diff --git a/PKG-INFO b/PKG-INFO index a0334767..b8768fbf 100644 --- a/PKG-INFO +++ b/PKG-INFO @@ -1,206 +1,206 @@ Metadata-Version: 2.1 Name: swh.web -Version: 0.0.371 +Version: 0.0.372 Summary: Software Heritage Web UI Home-page: https://forge.softwareheritage.org/diffusion/DWUI/ Author: Software Heritage developers Author-email: swh-devel@inria.fr License: UNKNOWN Project-URL: Bug Reports, https://forge.softwareheritage.org/maniphest Project-URL: Funding, https://www.softwareheritage.org/donate Project-URL: Source, https://forge.softwareheritage.org/source/swh-web Project-URL: Documentation, https://docs.softwareheritage.org/devel/swh-web/ Platform: UNKNOWN Classifier: Programming Language :: Python :: 3 Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+) Classifier: Operating System :: OS Independent Classifier: Development Status :: 5 - Production/Stable Classifier: Framework :: Django Requires-Python: >=3.7 Description-Content-Type: text/markdown Provides-Extra: testing License-File: LICENSE License-File: AUTHORS # swh-web This repository holds the development of Software Heritage web applications: * swh-web API (https://archive.softwareheritage.org/api): enables to query the content of the archive through HTTP requests and get responses in JSON or YAML. * swh-web browse (https://archive.softwareheritage.org/browse): graphical interface that eases the navigation in the archive. Documentation about how to use these components but also the details of their URI schemes can be found in the docs folder. The produced HTML documentation can be read and browsed at https://docs.softwareheritage.org/devel/swh-web/index.html. ## Technical details Those applications are powered by: * [Django Web Framework](https://www.djangoproject.com/) on the backend side with the following extensions enabled: * [django-rest-framework](http://www.django-rest-framework.org/) * [django-webpack-loader](https://github.com/owais/django-webpack-loader) * [django-js-reverse](http://django-js-reverse.readthedocs.io/en/latest/) * [webpack](https://webpack.js.org/) on the frontend side for better static assets management, including: * assets dependencies management and retrieval through [yarn](https://yarnpkg.com/en/) * linting of custom javascript code (through [eslint](https://eslint.org/)) and stylesheets (through [stylelint](https://stylelint.io/)) * use of [es6](http://es6-features.org) syntax and advanced javascript feature like [async/await](https://javascript.info/async-await) or [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) thanks to [babel](https://babeljs.io/) (es6 to es5 transpiler and polyfills provider) * assets minification (using [terser](https://github.com/terser-js/terser) and [cssnano](http://cssnano.co/)) but also dead code elimination for production use ## How to build, run and test ### Backend requirements First you will need [Python 3](https://www.python.org) and a complete [swh development environment](https://forge.softwareheritage.org/source/swh-environment/) installed. To run the backend, you need to have the following [Python 3 modules](requirements.txt) installed. To run the backend tests, the following [Python 3 modules](requirements-test.txt) are also required to be installed. One easy way to install them is to use the `pip` tool: ``` $ pip install -r requirements.txt -r requirements-test.txt ``` ### Frontend requirements To compile the frontend assets, you need to have [nodejs](https://nodejs.org/en/) >= 12.0.0 and [yarn](https://yarnpkg.com/en/) installed. If you are on Debian, you can easily install an up to date nodejs from the [nodesource](https://github.com/nodesource/distributions/blob/master/README.md) repository. Packages for yarn can be installed by following [these instructions](https://yarnpkg.com/en/docs/install#debian-stable). Alternatively, you can install yarn with `npm install yarn`, and add `YARN=node_modules/yarn/bin/yarn` as argument whenever you run `make`. Please note that the static assets bundles generated by webpack are not stored in the git repository. Follow the instructions below in order to generate them in order to be able to run the frontend part of the web applications. ### Make targets to execute the applications Below is the list of available make targets that can be executed from the root directory of swh-web in order to build and/or execute the web applications under various configurations: * **run-django-webpack-devserver**: Compile and serve not optimized (without mignification and dead code elimination) frontend static assets using [webpack-dev-server](https://github.com/webpack/webpack-dev-server) and run django server with development settings. This is the recommended target to use when developing swh-web as it enables automatic reloading of backend and frontend part of the applications when modifying source files (*.py, *.js, *.css, *.html). * **run-django-webpack-dev**: Compile not optimized (no minification, no dead code elimination) frontend static assets using webpack and run django server with development settings. This is the recommended target when one only wants to develop the backend side of the application. * **run-django-webpack-prod**: Compile optimized (with minification and dead code elimination) frontend static assets using webpack and run django server with production settings. This is useful to test the applications in production mode (with the difference that static assets are served by django). Production settings notably enable advanced django caching and you will need to have [memcached](https://memcached.org/) installed for that feature to work. * **run-django-server-dev**: Run the django server with development settings but without compiling frontend static assets through webpack. * **run-django-server-prod**: Run the django server with production settings but without compiling frontend static assets through webpack. * **run-gunicorn-server**: Run the web applications with production settings in a [gunicorn](http://gunicorn.org/) worker as they will be in real production environment. Once one of these targets executed, the web applications can be executed by pointing your browser to http://localhost:5004. ### Make targets to test the applications Some make targets are also available to easily execute the backend and frontend tests of the Software Heritage web applications. The backend tests are powered by the [pytest](https://docs.pytest.org/en/latest/) and [hypothesis](https://hypothesis.readthedocs.io/en/latest/) frameworks while the frontend ones rely on the use of the [cypress](https://www.cypress.io/) tool. Below is the exhaustive list of those targets: * **test**: execute the backend tests using a fast hypothesis profile (only one input example will be provided for each test) * **test-full**: execute the backend tests using a slower hypothesis profile (one hundred of input examples will be provided for each test which helps spotting possible bugs) * **test-frontend**: execute the frontend tests using cypress in headless mode but with some slow test suites disabled * **test-frontend-full**: execute the frontend tests using cypress in headless mode with all test suites enabled * **test-frontend-ui**: execute the frontend tests using the cypress GUI but with some slow test suites disabled * **test-frontend-full-ui**: execute the frontend tests using the cypress GUI with all test suites enabled ### Yarn targets Below is a list of available yarn targets in order to only execute the frontend static assets compilation (no web server will be executed): * **build-dev**: compile not optimized (without mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. * **build**: compile optimized (with mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. **The build target must be executed prior performing the Debian packaging of swh-web** in order for the package to contain the optimized assets dedicated to production environment. To execute these targets, issue the following command: ``` $ yarn ``` diff --git a/requirements.txt b/requirements.txt index b22d175b..08c82a90 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,30 +1,36 @@ # Add here external Python modules dependencies, one per line. Module names # should match https://pypi.python.org/pypi names. For the full spec or # dependency lines, see https://pip.readthedocs.org/en/1.1/requirements.html beautifulsoup4 chardet cryptography django < 3 django-cors-headers django-js-reverse djangorestframework django-webpack-loader docutils htmlmin iso8601 lxml -# Fix 'ImportError: cannot import name 'soft_unicode' from 'markupsafe'' -# when running pytest -# TODO: remove that dependency once pybadges depends on Jinja2 >=3 -markupsafe < 2.1.0 prometheus-client pybadges >= 2.2.1 pygments python-magic >= 0.4.0 python-memcached pyyaml requests sentry-sdk typing-extensions psycopg2 < 2.9 + +# TODO: remove dependencies below once pybadges depends on Jinja2 >=3 + +# Fix "ImportError: cannot import name 'soft_unicode' from 'markupsafe'" +# when running pytest +markupsafe < 2.1.0 +# Fix "ImportError: cannot import name 'json' from 'itsdangerous'" +# when running pytest after calling "./bin/install" from swh-environement +# in empty venv +flask >= 1.1.4 diff --git a/static/webpack-stats.json b/static/webpack-stats.json index 5252a989..a15731b9 100644 --- a/static/webpack-stats.json +++ b/static/webpack-stats.json @@ -1,776 +1,776 @@ { "status": "done", "assets": { "img/thirdParty/chosen-sprite.png": { "name": "img/thirdParty/chosen-sprite.png", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/img/thirdParty/chosen-sprite.png", "publicPath": "/static/img/thirdParty/chosen-sprite.png" }, "img/thirdParty/chosen-sprite@2x.png": { "name": "img/thirdParty/chosen-sprite@2x.png", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/img/thirdParty/chosen-sprite@2x.png", "publicPath": "/static/img/thirdParty/chosen-sprite@2x.png" }, "fonts/materialdesignicons-webfont.woff2?v=6.5.95": { "name": "fonts/materialdesignicons-webfont.woff2?v=6.5.95", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/materialdesignicons-webfont.woff2", "publicPath": "/static/fonts/materialdesignicons-webfont.woff2?v=6.5.95" }, - "fonts/materialdesignicons-webfont.woff?v=6.5.95": { - "name": "fonts/materialdesignicons-webfont.woff?v=6.5.95", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/materialdesignicons-webfont.woff", - "publicPath": "/static/fonts/materialdesignicons-webfont.woff?v=6.5.95" - }, "fonts/materialdesignicons-webfont.eot?v=6.5.95": { "name": "fonts/materialdesignicons-webfont.eot?v=6.5.95", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/materialdesignicons-webfont.eot", "publicPath": "/static/fonts/materialdesignicons-webfont.eot?v=6.5.95" }, "fonts/materialdesignicons-webfont.eot": { "name": "fonts/materialdesignicons-webfont.eot", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/materialdesignicons-webfont.eot", "publicPath": "/static/fonts/materialdesignicons-webfont.eot" }, + "fonts/materialdesignicons-webfont.woff?v=6.5.95": { + "name": "fonts/materialdesignicons-webfont.woff?v=6.5.95", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/materialdesignicons-webfont.woff", + "publicPath": "/static/fonts/materialdesignicons-webfont.woff?v=6.5.95" + }, "fonts/materialdesignicons-webfont.ttf?v=6.5.95": { "name": "fonts/materialdesignicons-webfont.ttf?v=6.5.95", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/materialdesignicons-webfont.ttf", "publicPath": "/static/fonts/materialdesignicons-webfont.ttf?v=6.5.95" }, "fonts/alegreya-latin-400.woff2": { "name": "fonts/alegreya-latin-400.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-400.woff2", "publicPath": "/static/fonts/alegreya-latin-400.woff2" }, "fonts/alegreya-latin-400.woff": { "name": "fonts/alegreya-latin-400.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-400.woff", "publicPath": "/static/fonts/alegreya-latin-400.woff" }, "fonts/alegreya-latin-400italic.woff2": { "name": "fonts/alegreya-latin-400italic.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-400italic.woff2", "publicPath": "/static/fonts/alegreya-latin-400italic.woff2" }, "fonts/alegreya-latin-400italic.woff": { "name": "fonts/alegreya-latin-400italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-400italic.woff", "publicPath": "/static/fonts/alegreya-latin-400italic.woff" }, "fonts/alegreya-latin-500.woff2": { "name": "fonts/alegreya-latin-500.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-500.woff2", "publicPath": "/static/fonts/alegreya-latin-500.woff2" }, - "fonts/alegreya-latin-500.woff": { - "name": "fonts/alegreya-latin-500.woff", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-500.woff", - "publicPath": "/static/fonts/alegreya-latin-500.woff" - }, "fonts/alegreya-latin-500italic.woff2": { "name": "fonts/alegreya-latin-500italic.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-500italic.woff2", "publicPath": "/static/fonts/alegreya-latin-500italic.woff2" }, + "fonts/alegreya-latin-500.woff": { + "name": "fonts/alegreya-latin-500.woff", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-500.woff", + "publicPath": "/static/fonts/alegreya-latin-500.woff" + }, "fonts/alegreya-latin-500italic.woff": { "name": "fonts/alegreya-latin-500italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-500italic.woff", "publicPath": "/static/fonts/alegreya-latin-500italic.woff" }, "fonts/alegreya-latin-700.woff2": { "name": "fonts/alegreya-latin-700.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-700.woff2", "publicPath": "/static/fonts/alegreya-latin-700.woff2" }, "fonts/alegreya-latin-700.woff": { "name": "fonts/alegreya-latin-700.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-700.woff", "publicPath": "/static/fonts/alegreya-latin-700.woff" }, - "fonts/alegreya-latin-700italic.woff2": { - "name": "fonts/alegreya-latin-700italic.woff2", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-700italic.woff2", - "publicPath": "/static/fonts/alegreya-latin-700italic.woff2" - }, "fonts/alegreya-latin-700italic.woff": { "name": "fonts/alegreya-latin-700italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-700italic.woff", "publicPath": "/static/fonts/alegreya-latin-700italic.woff" }, + "fonts/alegreya-latin-700italic.woff2": { + "name": "fonts/alegreya-latin-700italic.woff2", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-700italic.woff2", + "publicPath": "/static/fonts/alegreya-latin-700italic.woff2" + }, "fonts/alegreya-latin-800.woff2": { "name": "fonts/alegreya-latin-800.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-800.woff2", "publicPath": "/static/fonts/alegreya-latin-800.woff2" }, + "fonts/alegreya-latin-800.woff": { + "name": "fonts/alegreya-latin-800.woff", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-800.woff", + "publicPath": "/static/fonts/alegreya-latin-800.woff" + }, "fonts/alegreya-latin-800italic.woff2": { "name": "fonts/alegreya-latin-800italic.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-800italic.woff2", "publicPath": "/static/fonts/alegreya-latin-800italic.woff2" }, - "fonts/alegreya-latin-800italic.woff": { - "name": "fonts/alegreya-latin-800italic.woff", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-800italic.woff", - "publicPath": "/static/fonts/alegreya-latin-800italic.woff" - }, "fonts/alegreya-latin-900.woff": { "name": "fonts/alegreya-latin-900.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-900.woff", "publicPath": "/static/fonts/alegreya-latin-900.woff" }, + "fonts/alegreya-latin-800italic.woff": { + "name": "fonts/alegreya-latin-800italic.woff", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-800italic.woff", + "publicPath": "/static/fonts/alegreya-latin-800italic.woff" + }, "fonts/alegreya-latin-900.woff2": { "name": "fonts/alegreya-latin-900.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-900.woff2", "publicPath": "/static/fonts/alegreya-latin-900.woff2" }, - "fonts/alegreya-latin-800.woff": { - "name": "fonts/alegreya-latin-800.woff", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-800.woff", - "publicPath": "/static/fonts/alegreya-latin-800.woff" - }, "fonts/alegreya-latin-900italic.woff2": { "name": "fonts/alegreya-latin-900italic.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-900italic.woff2", "publicPath": "/static/fonts/alegreya-latin-900italic.woff2" }, "fonts/alegreya-latin-900italic.woff": { "name": "fonts/alegreya-latin-900italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-latin-900italic.woff", "publicPath": "/static/fonts/alegreya-latin-900italic.woff" }, "fonts/alegreya-sans-latin-100.woff2": { "name": "fonts/alegreya-sans-latin-100.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-100.woff2", "publicPath": "/static/fonts/alegreya-sans-latin-100.woff2" }, "fonts/alegreya-sans-latin-100.woff": { "name": "fonts/alegreya-sans-latin-100.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-100.woff", "publicPath": "/static/fonts/alegreya-sans-latin-100.woff" }, + "fonts/alegreya-sans-latin-100italic.woff2": { + "name": "fonts/alegreya-sans-latin-100italic.woff2", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-100italic.woff2", + "publicPath": "/static/fonts/alegreya-sans-latin-100italic.woff2" + }, "fonts/alegreya-sans-latin-100italic.woff": { "name": "fonts/alegreya-sans-latin-100italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-100italic.woff", "publicPath": "/static/fonts/alegreya-sans-latin-100italic.woff" }, "fonts/alegreya-sans-latin-300.woff2": { "name": "fonts/alegreya-sans-latin-300.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-300.woff2", "publicPath": "/static/fonts/alegreya-sans-latin-300.woff2" }, "fonts/alegreya-sans-latin-300.woff": { "name": "fonts/alegreya-sans-latin-300.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-300.woff", "publicPath": "/static/fonts/alegreya-sans-latin-300.woff" }, "fonts/alegreya-sans-latin-300italic.woff2": { "name": "fonts/alegreya-sans-latin-300italic.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-300italic.woff2", "publicPath": "/static/fonts/alegreya-sans-latin-300italic.woff2" }, - "fonts/alegreya-sans-latin-100italic.woff2": { - "name": "fonts/alegreya-sans-latin-100italic.woff2", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-100italic.woff2", - "publicPath": "/static/fonts/alegreya-sans-latin-100italic.woff2" - }, "fonts/alegreya-sans-latin-300italic.woff": { "name": "fonts/alegreya-sans-latin-300italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-300italic.woff", "publicPath": "/static/fonts/alegreya-sans-latin-300italic.woff" }, "fonts/alegreya-sans-latin-400.woff2": { "name": "fonts/alegreya-sans-latin-400.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-400.woff2", "publicPath": "/static/fonts/alegreya-sans-latin-400.woff2" }, - "fonts/alegreya-sans-latin-400.woff": { - "name": "fonts/alegreya-sans-latin-400.woff", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-400.woff", - "publicPath": "/static/fonts/alegreya-sans-latin-400.woff" - }, "fonts/alegreya-sans-latin-400italic.woff2": { "name": "fonts/alegreya-sans-latin-400italic.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-400italic.woff2", "publicPath": "/static/fonts/alegreya-sans-latin-400italic.woff2" }, + "fonts/alegreya-sans-latin-400.woff": { + "name": "fonts/alegreya-sans-latin-400.woff", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-400.woff", + "publicPath": "/static/fonts/alegreya-sans-latin-400.woff" + }, "fonts/alegreya-sans-latin-400italic.woff": { "name": "fonts/alegreya-sans-latin-400italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-400italic.woff", "publicPath": "/static/fonts/alegreya-sans-latin-400italic.woff" }, + "fonts/alegreya-sans-latin-500.woff2": { + "name": "fonts/alegreya-sans-latin-500.woff2", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-500.woff2", + "publicPath": "/static/fonts/alegreya-sans-latin-500.woff2" + }, "fonts/alegreya-sans-latin-500.woff": { "name": "fonts/alegreya-sans-latin-500.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-500.woff", "publicPath": "/static/fonts/alegreya-sans-latin-500.woff" }, "fonts/alegreya-sans-latin-500italic.woff2": { "name": "fonts/alegreya-sans-latin-500italic.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-500italic.woff2", "publicPath": "/static/fonts/alegreya-sans-latin-500italic.woff2" }, - "fonts/alegreya-sans-latin-700.woff2": { - "name": "fonts/alegreya-sans-latin-700.woff2", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-700.woff2", - "publicPath": "/static/fonts/alegreya-sans-latin-700.woff2" - }, - "fonts/alegreya-sans-latin-500.woff2": { - "name": "fonts/alegreya-sans-latin-500.woff2", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-500.woff2", - "publicPath": "/static/fonts/alegreya-sans-latin-500.woff2" - }, "fonts/alegreya-sans-latin-500italic.woff": { "name": "fonts/alegreya-sans-latin-500italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-500italic.woff", "publicPath": "/static/fonts/alegreya-sans-latin-500italic.woff" }, + "fonts/alegreya-sans-latin-700.woff2": { + "name": "fonts/alegreya-sans-latin-700.woff2", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-700.woff2", + "publicPath": "/static/fonts/alegreya-sans-latin-700.woff2" + }, "fonts/alegreya-sans-latin-700.woff": { "name": "fonts/alegreya-sans-latin-700.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-700.woff", "publicPath": "/static/fonts/alegreya-sans-latin-700.woff" }, - "fonts/alegreya-sans-latin-700italic.woff": { - "name": "fonts/alegreya-sans-latin-700italic.woff", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-700italic.woff", - "publicPath": "/static/fonts/alegreya-sans-latin-700italic.woff" - }, "fonts/alegreya-sans-latin-700italic.woff2": { "name": "fonts/alegreya-sans-latin-700italic.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-700italic.woff2", "publicPath": "/static/fonts/alegreya-sans-latin-700italic.woff2" }, + "fonts/alegreya-sans-latin-700italic.woff": { + "name": "fonts/alegreya-sans-latin-700italic.woff", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-700italic.woff", + "publicPath": "/static/fonts/alegreya-sans-latin-700italic.woff" + }, "fonts/alegreya-sans-latin-800.woff2": { "name": "fonts/alegreya-sans-latin-800.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-800.woff2", "publicPath": "/static/fonts/alegreya-sans-latin-800.woff2" }, "fonts/alegreya-sans-latin-800.woff": { "name": "fonts/alegreya-sans-latin-800.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-800.woff", "publicPath": "/static/fonts/alegreya-sans-latin-800.woff" }, "fonts/alegreya-sans-latin-800italic.woff2": { "name": "fonts/alegreya-sans-latin-800italic.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-800italic.woff2", "publicPath": "/static/fonts/alegreya-sans-latin-800italic.woff2" }, "fonts/alegreya-sans-latin-800italic.woff": { "name": "fonts/alegreya-sans-latin-800italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-800italic.woff", "publicPath": "/static/fonts/alegreya-sans-latin-800italic.woff" }, + "fonts/alegreya-sans-latin-900.woff2": { + "name": "fonts/alegreya-sans-latin-900.woff2", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-900.woff2", + "publicPath": "/static/fonts/alegreya-sans-latin-900.woff2" + }, "fonts/alegreya-sans-latin-900.woff": { "name": "fonts/alegreya-sans-latin-900.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-900.woff", "publicPath": "/static/fonts/alegreya-sans-latin-900.woff" }, "fonts/alegreya-sans-latin-900italic.woff2": { "name": "fonts/alegreya-sans-latin-900italic.woff2", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-900italic.woff2", "publicPath": "/static/fonts/alegreya-sans-latin-900italic.woff2" }, - "fonts/alegreya-sans-latin-900.woff2": { - "name": "fonts/alegreya-sans-latin-900.woff2", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-900.woff2", - "publicPath": "/static/fonts/alegreya-sans-latin-900.woff2" - }, "fonts/alegreya-sans-latin-900italic.woff": { "name": "fonts/alegreya-sans-latin-900italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/alegreya-sans-latin-900italic.woff", "publicPath": "/static/fonts/alegreya-sans-latin-900italic.woff" }, "js/pdf.worker.min.js": { "name": "js/pdf.worker.min.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/pdf.worker.min.js", "publicPath": "/static/js/pdf.worker.min.js" }, "fonts/MathJax_AMS-Regular.woff": { "name": "fonts/MathJax_AMS-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_AMS-Regular.woff", "publicPath": "/static/fonts/MathJax_AMS-Regular.woff" }, "fonts/MathJax_Calligraphic-Bold.woff": { "name": "fonts/MathJax_Calligraphic-Bold.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Calligraphic-Bold.woff", "publicPath": "/static/fonts/MathJax_Calligraphic-Bold.woff" }, "fonts/MathJax_Calligraphic-Regular.woff": { "name": "fonts/MathJax_Calligraphic-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Calligraphic-Regular.woff", "publicPath": "/static/fonts/MathJax_Calligraphic-Regular.woff" }, "fonts/MathJax_Fraktur-Bold.woff": { "name": "fonts/MathJax_Fraktur-Bold.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Fraktur-Bold.woff", "publicPath": "/static/fonts/MathJax_Fraktur-Bold.woff" }, "fonts/MathJax_Fraktur-Regular.woff": { "name": "fonts/MathJax_Fraktur-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Fraktur-Regular.woff", "publicPath": "/static/fonts/MathJax_Fraktur-Regular.woff" }, "fonts/MathJax_Main-Bold.woff": { "name": "fonts/MathJax_Main-Bold.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Main-Bold.woff", "publicPath": "/static/fonts/MathJax_Main-Bold.woff" }, "fonts/MathJax_Main-Italic.woff": { "name": "fonts/MathJax_Main-Italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Main-Italic.woff", "publicPath": "/static/fonts/MathJax_Main-Italic.woff" }, "fonts/MathJax_Main-Regular.woff": { "name": "fonts/MathJax_Main-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Main-Regular.woff", "publicPath": "/static/fonts/MathJax_Main-Regular.woff" }, "fonts/MathJax_Math-BoldItalic.woff": { "name": "fonts/MathJax_Math-BoldItalic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Math-BoldItalic.woff", "publicPath": "/static/fonts/MathJax_Math-BoldItalic.woff" }, "fonts/MathJax_Math-Italic.woff": { "name": "fonts/MathJax_Math-Italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Math-Italic.woff", "publicPath": "/static/fonts/MathJax_Math-Italic.woff" }, "fonts/MathJax_Math-Regular.woff": { "name": "fonts/MathJax_Math-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Math-Regular.woff", "publicPath": "/static/fonts/MathJax_Math-Regular.woff" }, "fonts/MathJax_SansSerif-Bold.woff": { "name": "fonts/MathJax_SansSerif-Bold.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_SansSerif-Bold.woff", "publicPath": "/static/fonts/MathJax_SansSerif-Bold.woff" }, "fonts/MathJax_SansSerif-Italic.woff": { "name": "fonts/MathJax_SansSerif-Italic.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_SansSerif-Italic.woff", "publicPath": "/static/fonts/MathJax_SansSerif-Italic.woff" }, "fonts/MathJax_SansSerif-Regular.woff": { "name": "fonts/MathJax_SansSerif-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_SansSerif-Regular.woff", "publicPath": "/static/fonts/MathJax_SansSerif-Regular.woff" }, "fonts/MathJax_Script-Regular.woff": { "name": "fonts/MathJax_Script-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Script-Regular.woff", "publicPath": "/static/fonts/MathJax_Script-Regular.woff" }, "fonts/MathJax_Size1-Regular.woff": { "name": "fonts/MathJax_Size1-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Size1-Regular.woff", "publicPath": "/static/fonts/MathJax_Size1-Regular.woff" }, "fonts/MathJax_Size2-Regular.woff": { "name": "fonts/MathJax_Size2-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Size2-Regular.woff", "publicPath": "/static/fonts/MathJax_Size2-Regular.woff" }, "fonts/MathJax_Size3-Regular.woff": { "name": "fonts/MathJax_Size3-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Size3-Regular.woff", "publicPath": "/static/fonts/MathJax_Size3-Regular.woff" }, "fonts/MathJax_Size4-Regular.woff": { "name": "fonts/MathJax_Size4-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Size4-Regular.woff", "publicPath": "/static/fonts/MathJax_Size4-Regular.woff" }, "fonts/MathJax_Typewriter-Regular.woff": { "name": "fonts/MathJax_Typewriter-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Typewriter-Regular.woff", "publicPath": "/static/fonts/MathJax_Typewriter-Regular.woff" }, "fonts/MathJax_Vector-Bold.woff": { "name": "fonts/MathJax_Vector-Bold.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Vector-Bold.woff", "publicPath": "/static/fonts/MathJax_Vector-Bold.woff" }, "fonts/MathJax_Vector-Regular.woff": { "name": "fonts/MathJax_Vector-Regular.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Vector-Regular.woff", "publicPath": "/static/fonts/MathJax_Vector-Regular.woff" }, "fonts/MathJax_Zero.woff": { "name": "fonts/MathJax_Zero.woff", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/fonts/MathJax_Zero.woff", "publicPath": "/static/fonts/MathJax_Zero.woff" }, "robots.txt": { "name": "robots.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/robots.txt", "publicPath": "/static/robots.txt" }, "js/pdf.worker.min.js.LICENSE.txt": { "name": "js/pdf.worker.min.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/pdf.worker.min.js.LICENSE.txt", "publicPath": "/static/js/pdf.worker.min.js.LICENSE.txt" }, "js/admin.dd251562dae83eb019a5.js.LICENSE.txt": { "name": "js/admin.dd251562dae83eb019a5.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/admin.dd251562dae83eb019a5.js.LICENSE.txt", "publicPath": "/static/js/admin.dd251562dae83eb019a5.js.LICENSE.txt" }, "js/auth.7381ff7d9581af98cd05.js.LICENSE.txt": { "name": "js/auth.7381ff7d9581af98cd05.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/auth.7381ff7d9581af98cd05.js.LICENSE.txt", "publicPath": "/static/js/auth.7381ff7d9581af98cd05.js.LICENSE.txt" }, "js/browse.85d04de230b236eacffa.js.LICENSE.txt": { "name": "js/browse.85d04de230b236eacffa.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/browse.85d04de230b236eacffa.js.LICENSE.txt", "publicPath": "/static/js/browse.85d04de230b236eacffa.js.LICENSE.txt" }, "js/guided_tour.4097532854c102d9725b.js.LICENSE.txt": { "name": "js/guided_tour.4097532854c102d9725b.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/guided_tour.4097532854c102d9725b.js.LICENSE.txt", "publicPath": "/static/js/guided_tour.4097532854c102d9725b.js.LICENSE.txt" }, "js/highlightjs.aef2297de7c1918965ad.js.LICENSE.txt": { "name": "js/highlightjs.aef2297de7c1918965ad.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/highlightjs.aef2297de7c1918965ad.js.LICENSE.txt", "publicPath": "/static/js/highlightjs.aef2297de7c1918965ad.js.LICENSE.txt" }, "js/revision.34adb35bf15ccf6957ef.js.LICENSE.txt": { "name": "js/revision.34adb35bf15ccf6957ef.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/revision.34adb35bf15ccf6957ef.js.LICENSE.txt", "publicPath": "/static/js/revision.34adb35bf15ccf6957ef.js.LICENSE.txt" }, "js/save.a2b0cf6fa1d51569fde2.js.LICENSE.txt": { "name": "js/save.a2b0cf6fa1d51569fde2.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/save.a2b0cf6fa1d51569fde2.js.LICENSE.txt", "publicPath": "/static/js/save.a2b0cf6fa1d51569fde2.js.LICENSE.txt" }, "js/showdown.482577f15a86ac41c94d.js.LICENSE.txt": { "name": "js/showdown.482577f15a86ac41c94d.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/showdown.482577f15a86ac41c94d.js.LICENSE.txt", "publicPath": "/static/js/showdown.482577f15a86ac41c94d.js.LICENSE.txt" }, "js/vault.147cdb83270419445ab0.js.LICENSE.txt": { "name": "js/vault.147cdb83270419445ab0.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/vault.147cdb83270419445ab0.js.LICENSE.txt", "publicPath": "/static/js/vault.147cdb83270419445ab0.js.LICENSE.txt" }, "js/vendors.2d7aafd4730af8b6e4c7.js.LICENSE.txt": { "name": "js/vendors.2d7aafd4730af8b6e4c7.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/vendors.2d7aafd4730af8b6e4c7.js.LICENSE.txt", "publicPath": "/static/js/vendors.2d7aafd4730af8b6e4c7.js.LICENSE.txt" }, "js/webapp.2c626e5a143ac60b4007.js.LICENSE.txt": { "name": "js/webapp.2c626e5a143ac60b4007.js.LICENSE.txt", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/webapp.2c626e5a143ac60b4007.js.LICENSE.txt", "publicPath": "/static/js/webapp.2c626e5a143ac60b4007.js.LICENSE.txt" }, "js/admin.dd251562dae83eb019a5.js": { "name": "js/admin.dd251562dae83eb019a5.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/admin.dd251562dae83eb019a5.js", "publicPath": "/static/js/admin.dd251562dae83eb019a5.js" }, "css/auth.0336a94c2c02b4b2a4f4.css": { "name": "css/auth.0336a94c2c02b4b2a4f4.css", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/auth.0336a94c2c02b4b2a4f4.css", "publicPath": "/static/css/auth.0336a94c2c02b4b2a4f4.css" }, "js/auth.7381ff7d9581af98cd05.js": { "name": "js/auth.7381ff7d9581af98cd05.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/auth.7381ff7d9581af98cd05.js", "publicPath": "/static/js/auth.7381ff7d9581af98cd05.js" }, "css/browse.6315ef52ed73df532bed.css": { "name": "css/browse.6315ef52ed73df532bed.css", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/browse.6315ef52ed73df532bed.css", "publicPath": "/static/css/browse.6315ef52ed73df532bed.css" }, "js/browse.85d04de230b236eacffa.js": { "name": "js/browse.85d04de230b236eacffa.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/browse.85d04de230b236eacffa.js", "publicPath": "/static/js/browse.85d04de230b236eacffa.js" }, "css/guided_tour.f2ef4d4b51ea7a882847.css": { "name": "css/guided_tour.f2ef4d4b51ea7a882847.css", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/guided_tour.f2ef4d4b51ea7a882847.css", "publicPath": "/static/css/guided_tour.f2ef4d4b51ea7a882847.css" }, "js/guided_tour.4097532854c102d9725b.js": { "name": "js/guided_tour.4097532854c102d9725b.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/guided_tour.4097532854c102d9725b.js", "publicPath": "/static/js/guided_tour.4097532854c102d9725b.js" }, "css/origin.5b45e9e6e54fd51ee886.css": { "name": "css/origin.5b45e9e6e54fd51ee886.css", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/origin.5b45e9e6e54fd51ee886.css", "publicPath": "/static/css/origin.5b45e9e6e54fd51ee886.css" }, "js/origin.c6ac2c3fd8c3ba8bc3d6.js": { "name": "js/origin.c6ac2c3fd8c3ba8bc3d6.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/origin.c6ac2c3fd8c3ba8bc3d6.js", "publicPath": "/static/js/origin.c6ac2c3fd8c3ba8bc3d6.js" }, "css/revision.5ddd36d69e1760bfa29d.css": { "name": "css/revision.5ddd36d69e1760bfa29d.css", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/revision.5ddd36d69e1760bfa29d.css", "publicPath": "/static/css/revision.5ddd36d69e1760bfa29d.css" }, "js/revision.34adb35bf15ccf6957ef.js": { "name": "js/revision.34adb35bf15ccf6957ef.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/revision.34adb35bf15ccf6957ef.js", "publicPath": "/static/js/revision.34adb35bf15ccf6957ef.js" }, "js/save.a2b0cf6fa1d51569fde2.js": { "name": "js/save.a2b0cf6fa1d51569fde2.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/save.a2b0cf6fa1d51569fde2.js", "publicPath": "/static/js/save.a2b0cf6fa1d51569fde2.js" }, "css/vault.25fc5883f848b48ffa5b.css": { "name": "css/vault.25fc5883f848b48ffa5b.css", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/vault.25fc5883f848b48ffa5b.css", "publicPath": "/static/css/vault.25fc5883f848b48ffa5b.css" }, "js/vault.147cdb83270419445ab0.js": { "name": "js/vault.147cdb83270419445ab0.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/vault.147cdb83270419445ab0.js", "publicPath": "/static/js/vault.147cdb83270419445ab0.js" }, "css/vendors.0dfd46f0c48f7ea5922b.css": { "name": "css/vendors.0dfd46f0c48f7ea5922b.css", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/vendors.0dfd46f0c48f7ea5922b.css", "publicPath": "/static/css/vendors.0dfd46f0c48f7ea5922b.css" }, "js/vendors.2d7aafd4730af8b6e4c7.js": { "name": "js/vendors.2d7aafd4730af8b6e4c7.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/vendors.2d7aafd4730af8b6e4c7.js", "publicPath": "/static/js/vendors.2d7aafd4730af8b6e4c7.js" }, "css/webapp.524d6dc01c6a847b4d8d.css": { "name": "css/webapp.524d6dc01c6a847b4d8d.css", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/webapp.524d6dc01c6a847b4d8d.css", "publicPath": "/static/css/webapp.524d6dc01c6a847b4d8d.css" }, "js/webapp.2c626e5a143ac60b4007.js": { "name": "js/webapp.2c626e5a143ac60b4007.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/webapp.2c626e5a143ac60b4007.js", "publicPath": "/static/js/webapp.2c626e5a143ac60b4007.js" }, "js/d3.9fbc8c4f808d6305db8c.js": { "name": "js/d3.9fbc8c4f808d6305db8c.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/d3.9fbc8c4f808d6305db8c.js", "publicPath": "/static/js/d3.9fbc8c4f808d6305db8c.js" }, "css/highlightjs.ae43064ab38a65a04d81.css": { "name": "css/highlightjs.ae43064ab38a65a04d81.css", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/highlightjs.ae43064ab38a65a04d81.css", "publicPath": "/static/css/highlightjs.ae43064ab38a65a04d81.css" }, "js/highlightjs.aef2297de7c1918965ad.js": { "name": "js/highlightjs.aef2297de7c1918965ad.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/highlightjs.aef2297de7c1918965ad.js", "publicPath": "/static/js/highlightjs.aef2297de7c1918965ad.js" }, "css/showdown.426fbf6a7a6653fd4cbb.css": { "name": "css/showdown.426fbf6a7a6653fd4cbb.css", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/showdown.426fbf6a7a6653fd4cbb.css", "publicPath": "/static/css/showdown.426fbf6a7a6653fd4cbb.css" }, "js/showdown.482577f15a86ac41c94d.js": { "name": "js/showdown.482577f15a86ac41c94d.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/showdown.482577f15a86ac41c94d.js", "publicPath": "/static/js/showdown.482577f15a86ac41c94d.js" }, "css/org.6851b70c924e28f6bf51.css": { "name": "css/org.6851b70c924e28f6bf51.css", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/org.6851b70c924e28f6bf51.css", "publicPath": "/static/css/org.6851b70c924e28f6bf51.css" }, "js/org.27a44f19269b37fa5fe6.js": { "name": "js/org.27a44f19269b37fa5fe6.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/org.27a44f19269b37fa5fe6.js", "publicPath": "/static/js/org.27a44f19269b37fa5fe6.js" }, "js/pdfjs.0de3de7f552746c41e31.js": { "name": "js/pdfjs.0de3de7f552746c41e31.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/pdfjs.0de3de7f552746c41e31.js", "publicPath": "/static/js/pdfjs.0de3de7f552746c41e31.js" }, "js/mathjax.82b6ebf86da778cadb7d.js": { "name": "js/mathjax.82b6ebf86da778cadb7d.js", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/mathjax.82b6ebf86da778cadb7d.js", "publicPath": "/static/js/mathjax.82b6ebf86da778cadb7d.js" }, "css/auth.0336a94c2c02b4b2a4f4.css.map": { "name": "css/auth.0336a94c2c02b4b2a4f4.css.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/auth.0336a94c2c02b4b2a4f4.css.map", "publicPath": "/static/css/auth.0336a94c2c02b4b2a4f4.css.map" }, - "css/guided_tour.f2ef4d4b51ea7a882847.css.map": { - "name": "css/guided_tour.f2ef4d4b51ea7a882847.css.map", - "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/guided_tour.f2ef4d4b51ea7a882847.css.map", - "publicPath": "/static/css/guided_tour.f2ef4d4b51ea7a882847.css.map" - }, "css/browse.6315ef52ed73df532bed.css.map": { "name": "css/browse.6315ef52ed73df532bed.css.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/browse.6315ef52ed73df532bed.css.map", "publicPath": "/static/css/browse.6315ef52ed73df532bed.css.map" }, "css/origin.5b45e9e6e54fd51ee886.css.map": { "name": "css/origin.5b45e9e6e54fd51ee886.css.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/origin.5b45e9e6e54fd51ee886.css.map", "publicPath": "/static/css/origin.5b45e9e6e54fd51ee886.css.map" }, + "css/guided_tour.f2ef4d4b51ea7a882847.css.map": { + "name": "css/guided_tour.f2ef4d4b51ea7a882847.css.map", + "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/guided_tour.f2ef4d4b51ea7a882847.css.map", + "publicPath": "/static/css/guided_tour.f2ef4d4b51ea7a882847.css.map" + }, "css/revision.5ddd36d69e1760bfa29d.css.map": { "name": "css/revision.5ddd36d69e1760bfa29d.css.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/revision.5ddd36d69e1760bfa29d.css.map", "publicPath": "/static/css/revision.5ddd36d69e1760bfa29d.css.map" }, "css/vault.25fc5883f848b48ffa5b.css.map": { "name": "css/vault.25fc5883f848b48ffa5b.css.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/vault.25fc5883f848b48ffa5b.css.map", "publicPath": "/static/css/vault.25fc5883f848b48ffa5b.css.map" }, "css/vendors.0dfd46f0c48f7ea5922b.css.map": { "name": "css/vendors.0dfd46f0c48f7ea5922b.css.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/vendors.0dfd46f0c48f7ea5922b.css.map", "publicPath": "/static/css/vendors.0dfd46f0c48f7ea5922b.css.map" }, "css/webapp.524d6dc01c6a847b4d8d.css.map": { "name": "css/webapp.524d6dc01c6a847b4d8d.css.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/webapp.524d6dc01c6a847b4d8d.css.map", "publicPath": "/static/css/webapp.524d6dc01c6a847b4d8d.css.map" }, "css/highlightjs.ae43064ab38a65a04d81.css.map": { "name": "css/highlightjs.ae43064ab38a65a04d81.css.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/highlightjs.ae43064ab38a65a04d81.css.map", "publicPath": "/static/css/highlightjs.ae43064ab38a65a04d81.css.map" }, "css/showdown.426fbf6a7a6653fd4cbb.css.map": { "name": "css/showdown.426fbf6a7a6653fd4cbb.css.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/showdown.426fbf6a7a6653fd4cbb.css.map", "publicPath": "/static/css/showdown.426fbf6a7a6653fd4cbb.css.map" }, "css/org.6851b70c924e28f6bf51.css.map": { "name": "css/org.6851b70c924e28f6bf51.css.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/css/org.6851b70c924e28f6bf51.css.map", "publicPath": "/static/css/org.6851b70c924e28f6bf51.css.map" }, "js/admin.dd251562dae83eb019a5.js.map": { "name": "js/admin.dd251562dae83eb019a5.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/admin.dd251562dae83eb019a5.js.map", "publicPath": "/static/js/admin.dd251562dae83eb019a5.js.map" }, "js/auth.7381ff7d9581af98cd05.js.map": { "name": "js/auth.7381ff7d9581af98cd05.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/auth.7381ff7d9581af98cd05.js.map", "publicPath": "/static/js/auth.7381ff7d9581af98cd05.js.map" }, "js/browse.85d04de230b236eacffa.js.map": { "name": "js/browse.85d04de230b236eacffa.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/browse.85d04de230b236eacffa.js.map", "publicPath": "/static/js/browse.85d04de230b236eacffa.js.map" }, "js/guided_tour.4097532854c102d9725b.js.map": { "name": "js/guided_tour.4097532854c102d9725b.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/guided_tour.4097532854c102d9725b.js.map", "publicPath": "/static/js/guided_tour.4097532854c102d9725b.js.map" }, "js/origin.c6ac2c3fd8c3ba8bc3d6.js.map": { "name": "js/origin.c6ac2c3fd8c3ba8bc3d6.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/origin.c6ac2c3fd8c3ba8bc3d6.js.map", "publicPath": "/static/js/origin.c6ac2c3fd8c3ba8bc3d6.js.map" }, "js/revision.34adb35bf15ccf6957ef.js.map": { "name": "js/revision.34adb35bf15ccf6957ef.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/revision.34adb35bf15ccf6957ef.js.map", "publicPath": "/static/js/revision.34adb35bf15ccf6957ef.js.map" }, "js/save.a2b0cf6fa1d51569fde2.js.map": { "name": "js/save.a2b0cf6fa1d51569fde2.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/save.a2b0cf6fa1d51569fde2.js.map", "publicPath": "/static/js/save.a2b0cf6fa1d51569fde2.js.map" }, "js/vault.147cdb83270419445ab0.js.map": { "name": "js/vault.147cdb83270419445ab0.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/vault.147cdb83270419445ab0.js.map", "publicPath": "/static/js/vault.147cdb83270419445ab0.js.map" }, "js/vendors.2d7aafd4730af8b6e4c7.js.map": { "name": "js/vendors.2d7aafd4730af8b6e4c7.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/vendors.2d7aafd4730af8b6e4c7.js.map", "publicPath": "/static/js/vendors.2d7aafd4730af8b6e4c7.js.map" }, "js/webapp.2c626e5a143ac60b4007.js.map": { "name": "js/webapp.2c626e5a143ac60b4007.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/webapp.2c626e5a143ac60b4007.js.map", "publicPath": "/static/js/webapp.2c626e5a143ac60b4007.js.map" }, "js/d3.9fbc8c4f808d6305db8c.js.map": { "name": "js/d3.9fbc8c4f808d6305db8c.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/d3.9fbc8c4f808d6305db8c.js.map", "publicPath": "/static/js/d3.9fbc8c4f808d6305db8c.js.map" }, "js/highlightjs.aef2297de7c1918965ad.js.map": { "name": "js/highlightjs.aef2297de7c1918965ad.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/highlightjs.aef2297de7c1918965ad.js.map", "publicPath": "/static/js/highlightjs.aef2297de7c1918965ad.js.map" }, "js/showdown.482577f15a86ac41c94d.js.map": { "name": "js/showdown.482577f15a86ac41c94d.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/showdown.482577f15a86ac41c94d.js.map", "publicPath": "/static/js/showdown.482577f15a86ac41c94d.js.map" }, "js/org.27a44f19269b37fa5fe6.js.map": { "name": "js/org.27a44f19269b37fa5fe6.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/org.27a44f19269b37fa5fe6.js.map", "publicPath": "/static/js/org.27a44f19269b37fa5fe6.js.map" }, "js/pdfjs.0de3de7f552746c41e31.js.map": { "name": "js/pdfjs.0de3de7f552746c41e31.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/pdfjs.0de3de7f552746c41e31.js.map", "publicPath": "/static/js/pdfjs.0de3de7f552746c41e31.js.map" }, "js/mathjax.82b6ebf86da778cadb7d.js.map": { "name": "js/mathjax.82b6ebf86da778cadb7d.js.map", "path": "/var/lib/jenkins/workspace/DWAPPS/pypi-upload/static/js/mathjax.82b6ebf86da778cadb7d.js.map", "publicPath": "/static/js/mathjax.82b6ebf86da778cadb7d.js.map" } }, "chunks": { "admin": [ "js/admin.dd251562dae83eb019a5.js" ], "auth": [ "css/auth.0336a94c2c02b4b2a4f4.css", "js/auth.7381ff7d9581af98cd05.js" ], "browse": [ "css/browse.6315ef52ed73df532bed.css", "js/browse.85d04de230b236eacffa.js" ], "guided_tour": [ "css/guided_tour.f2ef4d4b51ea7a882847.css", "js/guided_tour.4097532854c102d9725b.js" ], "origin": [ "css/origin.5b45e9e6e54fd51ee886.css", "js/origin.c6ac2c3fd8c3ba8bc3d6.js" ], "revision": [ "css/revision.5ddd36d69e1760bfa29d.css", "js/revision.34adb35bf15ccf6957ef.js" ], "save": [ "js/save.a2b0cf6fa1d51569fde2.js" ], "vault": [ "css/vault.25fc5883f848b48ffa5b.css", "js/vault.147cdb83270419445ab0.js" ], "vendors": [ "css/vendors.0dfd46f0c48f7ea5922b.css", "js/vendors.2d7aafd4730af8b6e4c7.js" ], "webapp": [ "css/webapp.524d6dc01c6a847b4d8d.css", "js/webapp.2c626e5a143ac60b4007.js" ] }, "publicPath": "/static/" } \ No newline at end of file diff --git a/swh.web.egg-info/PKG-INFO b/swh.web.egg-info/PKG-INFO index a0334767..b8768fbf 100644 --- a/swh.web.egg-info/PKG-INFO +++ b/swh.web.egg-info/PKG-INFO @@ -1,206 +1,206 @@ Metadata-Version: 2.1 Name: swh.web -Version: 0.0.371 +Version: 0.0.372 Summary: Software Heritage Web UI Home-page: https://forge.softwareheritage.org/diffusion/DWUI/ Author: Software Heritage developers Author-email: swh-devel@inria.fr License: UNKNOWN Project-URL: Bug Reports, https://forge.softwareheritage.org/maniphest Project-URL: Funding, https://www.softwareheritage.org/donate Project-URL: Source, https://forge.softwareheritage.org/source/swh-web Project-URL: Documentation, https://docs.softwareheritage.org/devel/swh-web/ Platform: UNKNOWN Classifier: Programming Language :: Python :: 3 Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+) Classifier: Operating System :: OS Independent Classifier: Development Status :: 5 - Production/Stable Classifier: Framework :: Django Requires-Python: >=3.7 Description-Content-Type: text/markdown Provides-Extra: testing License-File: LICENSE License-File: AUTHORS # swh-web This repository holds the development of Software Heritage web applications: * swh-web API (https://archive.softwareheritage.org/api): enables to query the content of the archive through HTTP requests and get responses in JSON or YAML. * swh-web browse (https://archive.softwareheritage.org/browse): graphical interface that eases the navigation in the archive. Documentation about how to use these components but also the details of their URI schemes can be found in the docs folder. The produced HTML documentation can be read and browsed at https://docs.softwareheritage.org/devel/swh-web/index.html. ## Technical details Those applications are powered by: * [Django Web Framework](https://www.djangoproject.com/) on the backend side with the following extensions enabled: * [django-rest-framework](http://www.django-rest-framework.org/) * [django-webpack-loader](https://github.com/owais/django-webpack-loader) * [django-js-reverse](http://django-js-reverse.readthedocs.io/en/latest/) * [webpack](https://webpack.js.org/) on the frontend side for better static assets management, including: * assets dependencies management and retrieval through [yarn](https://yarnpkg.com/en/) * linting of custom javascript code (through [eslint](https://eslint.org/)) and stylesheets (through [stylelint](https://stylelint.io/)) * use of [es6](http://es6-features.org) syntax and advanced javascript feature like [async/await](https://javascript.info/async-await) or [fetch](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) thanks to [babel](https://babeljs.io/) (es6 to es5 transpiler and polyfills provider) * assets minification (using [terser](https://github.com/terser-js/terser) and [cssnano](http://cssnano.co/)) but also dead code elimination for production use ## How to build, run and test ### Backend requirements First you will need [Python 3](https://www.python.org) and a complete [swh development environment](https://forge.softwareheritage.org/source/swh-environment/) installed. To run the backend, you need to have the following [Python 3 modules](requirements.txt) installed. To run the backend tests, the following [Python 3 modules](requirements-test.txt) are also required to be installed. One easy way to install them is to use the `pip` tool: ``` $ pip install -r requirements.txt -r requirements-test.txt ``` ### Frontend requirements To compile the frontend assets, you need to have [nodejs](https://nodejs.org/en/) >= 12.0.0 and [yarn](https://yarnpkg.com/en/) installed. If you are on Debian, you can easily install an up to date nodejs from the [nodesource](https://github.com/nodesource/distributions/blob/master/README.md) repository. Packages for yarn can be installed by following [these instructions](https://yarnpkg.com/en/docs/install#debian-stable). Alternatively, you can install yarn with `npm install yarn`, and add `YARN=node_modules/yarn/bin/yarn` as argument whenever you run `make`. Please note that the static assets bundles generated by webpack are not stored in the git repository. Follow the instructions below in order to generate them in order to be able to run the frontend part of the web applications. ### Make targets to execute the applications Below is the list of available make targets that can be executed from the root directory of swh-web in order to build and/or execute the web applications under various configurations: * **run-django-webpack-devserver**: Compile and serve not optimized (without mignification and dead code elimination) frontend static assets using [webpack-dev-server](https://github.com/webpack/webpack-dev-server) and run django server with development settings. This is the recommended target to use when developing swh-web as it enables automatic reloading of backend and frontend part of the applications when modifying source files (*.py, *.js, *.css, *.html). * **run-django-webpack-dev**: Compile not optimized (no minification, no dead code elimination) frontend static assets using webpack and run django server with development settings. This is the recommended target when one only wants to develop the backend side of the application. * **run-django-webpack-prod**: Compile optimized (with minification and dead code elimination) frontend static assets using webpack and run django server with production settings. This is useful to test the applications in production mode (with the difference that static assets are served by django). Production settings notably enable advanced django caching and you will need to have [memcached](https://memcached.org/) installed for that feature to work. * **run-django-server-dev**: Run the django server with development settings but without compiling frontend static assets through webpack. * **run-django-server-prod**: Run the django server with production settings but without compiling frontend static assets through webpack. * **run-gunicorn-server**: Run the web applications with production settings in a [gunicorn](http://gunicorn.org/) worker as they will be in real production environment. Once one of these targets executed, the web applications can be executed by pointing your browser to http://localhost:5004. ### Make targets to test the applications Some make targets are also available to easily execute the backend and frontend tests of the Software Heritage web applications. The backend tests are powered by the [pytest](https://docs.pytest.org/en/latest/) and [hypothesis](https://hypothesis.readthedocs.io/en/latest/) frameworks while the frontend ones rely on the use of the [cypress](https://www.cypress.io/) tool. Below is the exhaustive list of those targets: * **test**: execute the backend tests using a fast hypothesis profile (only one input example will be provided for each test) * **test-full**: execute the backend tests using a slower hypothesis profile (one hundred of input examples will be provided for each test which helps spotting possible bugs) * **test-frontend**: execute the frontend tests using cypress in headless mode but with some slow test suites disabled * **test-frontend-full**: execute the frontend tests using cypress in headless mode with all test suites enabled * **test-frontend-ui**: execute the frontend tests using the cypress GUI but with some slow test suites disabled * **test-frontend-full-ui**: execute the frontend tests using the cypress GUI with all test suites enabled ### Yarn targets Below is a list of available yarn targets in order to only execute the frontend static assets compilation (no web server will be executed): * **build-dev**: compile not optimized (without mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. * **build**: compile optimized (with mignification and dead code elimination) frontend static assets and store the results in the `swh/web/static` folder. **The build target must be executed prior performing the Debian packaging of swh-web** in order for the package to contain the optimized assets dedicated to production environment. To execute these targets, issue the following command: ``` $ yarn ``` diff --git a/swh.web.egg-info/requires.txt b/swh.web.egg-info/requires.txt index 8c1f675d..1869c24b 100644 --- a/swh.web.egg-info/requires.txt +++ b/swh.web.egg-info/requires.txt @@ -1,53 +1,54 @@ beautifulsoup4 chardet cryptography django<3 django-cors-headers django-js-reverse djangorestframework django-webpack-loader docutils htmlmin iso8601 lxml -markupsafe<2.1.0 prometheus-client pybadges>=2.2.1 pygments python-magic>=0.4.0 python-memcached pyyaml requests sentry-sdk typing-extensions psycopg2<2.9 +markupsafe<2.1.0 +flask>=1.1.4 swh.auth[django]>=0.5.3 swh.core>=0.0.95 swh.counters>=0.5.1 swh.indexer>=0.4.1 swh.model>=2.6.3 swh.scheduler>=0.7.0 swh.search>=0.2.0 swh.storage>=0.31.0 swh.vault>=1.0.0 [testing] decorator djangorestframework-stubs django-stubs django-test-migrations hypothesis pytest<7.0.0 pytest-django pytest-mock pytest-postgresql requests-mock!=1.9.0,!=1.9.1 swh.core[http]>=0.0.95 swh.loader.git>=0.8.0 swh-scheduler[testing]>=0.5.0 swh.storage>=0.1.1 types-chardet types-docutils types-psycopg2 types-pyyaml types-requests diff --git a/swh/web/common/swh_templatetags.py b/swh/web/common/swh_templatetags.py index 1edbd632..627b2050 100644 --- a/swh/web/common/swh_templatetags.py +++ b/swh/web/common/swh_templatetags.py @@ -1,153 +1,149 @@ # Copyright (C) 2017-2021 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU Affero General Public License version 3, or any later version # See top-level LICENSE file for more information import json import re from django import template from django.utils.safestring import mark_safe from swh.web.common.converters import SWHDjangoJSONEncoder from swh.web.common.origin_save import get_savable_visit_types from swh.web.common.utils import rst_to_html register = template.Library() @register.filter def docstring_display(docstring): """ Utility function to htmlize reST-formatted documentation in browsable api. """ return rst_to_html(docstring) @register.filter def urlize_links_and_mails(text): """Utility function for decorating api links in browsable api. Args: text: whose content matching links should be transformed into contextual API or Browse html links. Returns The text transformed if any link is found. The text as is otherwise. """ if 'href="' not in text: text = re.sub(r"(http.*)", r'\1', text) return re.sub(r'([^ <>"]+@[^ <>"]+)', r'\1', text) return text @register.filter def urlize_header_links(text): """Utility function for decorating headers links in browsable api. Args text: Text whose content contains Link header value Returns: The text transformed with html link if any link is found. The text as is otherwise. """ - links = text.split(",") - ret = "" - for i, link in enumerate(links): - ret += re.sub(r"<(http.*)>", r'<\1>', link) - # add one link per line and align them - if i != len(links) - 1: - ret += "\n " - return ret + ret = re.sub( + r'<(http[^<>]+)>; rel="([^,]+)"', r'<\1>; rel="\2"\n', text + ).replace("\n,", "\n") + return ret[:-1] @register.filter def jsonify(obj): """Utility function for converting a django template variable to JSON in order to use it in script tags. Args obj: Any django template context variable Returns: JSON representation of the variable. """ return mark_safe(json.dumps(obj, cls=SWHDjangoJSONEncoder)) @register.filter def sub(value, arg): """Django template filter for subtracting two numbers Args: value (int/float): the value to subtract from arg (int/float): the value to subtract to Returns: int/float: The subtraction result """ return value - arg @register.filter def mul(value, arg): """Django template filter for multiplying two numbers Args: value (int/float): the value to multiply from arg (int/float): the value to multiply with Returns: int/float: The multiplication result """ return value * arg @register.filter def key_value(dict, key): """Django template filter to get a value in a dictionary. Args: dict (dict): a dictionary key (str): the key to lookup value Returns: The requested value in the dictionary """ return dict[key] @register.filter def visit_type_savable(visit_type: str) -> bool: """Django template filter to check if a save request can be created for a given visit type. Args: visit_type: the type of visit Returns: If the visit type is saveable or not """ return visit_type in get_savable_visit_types() @register.filter def split(value, arg): """Django template filter to split a string. Args: value (str): the string to split arg (str): the split separator Returns: list: the split string parts """ return value.split(arg) diff --git a/swh/web/common/utils.py b/swh/web/common/utils.py index ca17202a..0ffc772d 100644 --- a/swh/web/common/utils.py +++ b/swh/web/common/utils.py @@ -1,429 +1,449 @@ # Copyright (C) 2017-2021 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU Affero General Public License version 3, or any later version # See top-level LICENSE file for more information from datetime import datetime, timezone import os import re from typing import Any, Dict, List, Optional import urllib.parse from bs4 import BeautifulSoup from docutils.core import publish_parts import docutils.parsers.rst import docutils.utils from docutils.writers.html5_polyglot import HTMLTranslator, Writer from iso8601 import ParseError, parse_date from pkg_resources import get_distribution from prometheus_client.registry import CollectorRegistry import requests from requests.auth import HTTPBasicAuth from django.core.cache import cache from django.http import HttpRequest, QueryDict from django.shortcuts import redirect from django.urls import resolve from django.urls import reverse as django_reverse from swh.web.auth.utils import ADMIN_LIST_DEPOSIT_PERMISSION from swh.web.common.exc import BadInputExc from swh.web.common.typing import QueryParameters -from swh.web.config import get_config, search +from swh.web.config import SWH_WEB_SERVER_NAME, get_config, search SWH_WEB_METRICS_REGISTRY = CollectorRegistry(auto_describe=True) swh_object_icons = { "alias": "mdi mdi-star", "branch": "mdi mdi-source-branch", "branches": "mdi mdi-source-branch", "content": "mdi mdi-file-document", "cnt": "mdi mdi-file-document", "directory": "mdi mdi-folder", "dir": "mdi mdi-folder", "origin": "mdi mdi-source-repository", "ori": "mdi mdi-source-repository", "person": "mdi mdi-account", "revisions history": "mdi mdi-history", "release": "mdi mdi-tag", "rel": "mdi mdi-tag", "releases": "mdi mdi-tag", "revision": "mdi mdi-rotate-90 mdi-source-commit", "rev": "mdi mdi-rotate-90 mdi-source-commit", "snapshot": "mdi mdi-camera", "snp": "mdi mdi-camera", "visits": "mdi mdi-calendar-month", } def reverse( viewname: str, url_args: Optional[Dict[str, Any]] = None, query_params: Optional[QueryParameters] = None, current_app: Optional[str] = None, urlconf: Optional[str] = None, request: Optional[HttpRequest] = None, ) -> str: """An override of django reverse function supporting query parameters. Args: viewname: the name of the django view from which to compute a url url_args: dictionary of url arguments indexed by their names query_params: dictionary of query parameters to append to the reversed url current_app: the name of the django app tighten to the view urlconf: url configuration module request: build an absolute URI if provided Returns: str: the url of the requested view with processed arguments and query parameters """ if url_args: url_args = {k: v for k, v in url_args.items() if v is not None} url = django_reverse( viewname, urlconf=urlconf, kwargs=url_args, current_app=current_app ) if query_params: query_params = {k: v for k, v in query_params.items() if v is not None} if query_params and len(query_params) > 0: query_dict = QueryDict("", mutable=True) for k in sorted(query_params.keys()): query_dict[k] = query_params[k] url += "?" + query_dict.urlencode(safe="/;:") if request is not None: url = request.build_absolute_uri(url) return url def datetime_to_utc(date): """Returns datetime in UTC without timezone info Args: date (datetime.datetime): input datetime with timezone info Returns: datetime.datetime: datetime in UTC without timezone info """ if date.tzinfo and date.tzinfo != timezone.utc: return date.astimezone(tz=timezone.utc) else: return date def parse_iso8601_date_to_utc(iso_date: str) -> datetime: """Given an ISO 8601 datetime string, parse the result as UTC datetime. Returns: a timezone-aware datetime representing the parsed date Raises: swh.web.common.exc.BadInputExc: provided date does not respect ISO 8601 format Samples: - 2016-01-12 - 2016-01-12T09:19:12+0100 - 2007-01-14T20:34:22Z """ try: date = parse_date(iso_date) return datetime_to_utc(date) except ParseError as e: raise BadInputExc(e) def shorten_path(path): """Shorten the given path: for each hash present, only return the first 8 characters followed by an ellipsis""" sha256_re = r"([0-9a-f]{8})[0-9a-z]{56}" sha1_re = r"([0-9a-f]{8})[0-9a-f]{32}" ret = re.sub(sha256_re, r"\1...", path) return re.sub(sha1_re, r"\1...", ret) def format_utc_iso_date(iso_date, fmt="%d %B %Y, %H:%M UTC"): """Turns a string representation of an ISO 8601 datetime string to UTC and format it into a more human readable one. For instance, from the following input string: '2017-05-04T13:27:13+02:00' the following one is returned: '04 May 2017, 11:27 UTC'. Custom format string may also be provided as parameter Args: iso_date (str): a string representation of an ISO 8601 date fmt (str): optional date formatting string Returns: str: a formatted string representation of the input iso date """ if not iso_date: return iso_date date = parse_iso8601_date_to_utc(iso_date) return date.strftime(fmt) def gen_path_info(path): """Function to generate path data navigation for use with a breadcrumb in the swh web ui. For instance, from a path /folder1/folder2/folder3, it returns the following list:: [{'name': 'folder1', 'path': 'folder1'}, {'name': 'folder2', 'path': 'folder1/folder2'}, {'name': 'folder3', 'path': 'folder1/folder2/folder3'}] Args: path: a filesystem path Returns: list: a list of path data for navigation as illustrated above. """ path_info = [] if path: sub_paths = path.strip("/").split("/") path_from_root = "" for p in sub_paths: path_from_root += "/" + p path_info.append({"name": p, "path": path_from_root.strip("/")}) return path_info def parse_rst(text, report_level=2): """ Parse a reStructuredText string with docutils. Args: text (str): string with reStructuredText markups in it report_level (int): level of docutils report messages to print (1 info 2 warning 3 error 4 severe 5 none) Returns: docutils.nodes.document: a parsed docutils document """ parser = docutils.parsers.rst.Parser() components = (docutils.parsers.rst.Parser,) settings = docutils.frontend.OptionParser( components=components ).get_default_values() settings.report_level = report_level document = docutils.utils.new_document("rst-doc", settings=settings) parser.parse(text, document) return document def get_client_ip(request): """ Return the client IP address from an incoming HTTP request. Args: request (django.http.HttpRequest): the incoming HTTP request Returns: str: The client IP address """ x_forwarded_for = request.META.get("HTTP_X_FORWARDED_FOR") if x_forwarded_for: ip = x_forwarded_for.split(",")[0] else: ip = request.META.get("REMOTE_ADDR") return ip +def is_swh_web_development(request: HttpRequest) -> bool: + """Indicate if we are running a development version of swh-web. + """ + site_base_url = request.build_absolute_uri("/") + return any( + host in site_base_url for host in ("localhost", "127.0.0.1", "testserver") + ) + + +def is_swh_web_staging(request: HttpRequest) -> bool: + """Indicate if we are running a staging version of swh-web. + """ + config = get_config() + site_base_url = request.build_absolute_uri("/") + return any( + server_name in site_base_url for server_name in config["staging_server_names"] + ) + + +def is_swh_web_production(request: HttpRequest) -> bool: + """Indicate if we are running the public production version of swh-web. + """ + return SWH_WEB_SERVER_NAME in request.build_absolute_uri("/") + + browsers_supported_image_mimes = set( [ "image/gif", "image/png", "image/jpeg", "image/bmp", "image/webp", "image/svg", "image/svg+xml", ] ) def context_processor(request): """ Django context processor used to inject variables in all swh-web templates. """ config = get_config() if ( hasattr(request, "user") and request.user.is_authenticated and not hasattr(request.user, "backend") ): # To avoid django.template.base.VariableDoesNotExist errors # when rendering templates when standard Django user is logged in. request.user.backend = "django.contrib.auth.backends.ModelBackend" - site_base_url = request.build_absolute_uri("/") + return { "swh_object_icons": swh_object_icons, "available_languages": None, "swh_client_config": config["client_config"], "oidc_enabled": bool(config["keycloak"]["server_url"]), "browsers_supported_image_mimes": browsers_supported_image_mimes, "keycloak": config["keycloak"], - "site_base_url": site_base_url, + "site_base_url": request.build_absolute_uri("/"), "DJANGO_SETTINGS_MODULE": os.environ["DJANGO_SETTINGS_MODULE"], "status": config["status"], - "swh_web_dev": "localhost" in site_base_url, - "swh_web_staging": any( - [ - server_name in site_base_url - for server_name in config["staging_server_names"] - ] - ), + "swh_web_dev": is_swh_web_development(request), + "swh_web_staging": is_swh_web_staging(request), "swh_web_version": get_distribution("swh.web").version, "iframe_mode": False, "ADMIN_LIST_DEPOSIT_PERMISSION": ADMIN_LIST_DEPOSIT_PERMISSION, } def resolve_branch_alias( snapshot: Dict[str, Any], branch: Optional[Dict[str, Any]] ) -> Optional[Dict[str, Any]]: """ Resolve branch alias in snapshot content. Args: snapshot: a full snapshot content branch: a branch alias contained in the snapshot Returns: The real snapshot branch that got aliased. """ while branch and branch["target_type"] == "alias": if branch["target"] in snapshot["branches"]: branch = snapshot["branches"][branch["target"]] else: from swh.web.common import archive snp = archive.lookup_snapshot( snapshot["id"], branches_from=branch["target"], branches_count=1 ) if snp and branch["target"] in snp["branches"]: branch = snp["branches"][branch["target"]] else: branch = None return branch class _NoHeaderHTMLTranslator(HTMLTranslator): """ Docutils translator subclass to customize the generation of HTML from reST-formatted docstrings """ def __init__(self, document): super().__init__(document) self.body_prefix = [] self.body_suffix = [] _HTML_WRITER = Writer() _HTML_WRITER.translator_class = _NoHeaderHTMLTranslator def rst_to_html(rst: str) -> str: """ Convert reStructuredText document into HTML. Args: rst: A string containing a reStructuredText document Returns: Body content of the produced HTML conversion. """ settings = { "initial_header_level": 2, "halt_level": 4, "traceback": True, } pp = publish_parts(rst, writer=_HTML_WRITER, settings_overrides=settings) return f'
{pp["html_body"]}
' def prettify_html(html: str) -> str: """ Prettify an HTML document. Args: html: Input HTML document Returns: The prettified HTML document """ return BeautifulSoup(html, "lxml").prettify() def _deposits_list_url( deposits_list_base_url: str, page_size: int, username: Optional[str] ) -> str: params = {"page_size": str(page_size)} if username is not None: params["username"] = username return f"{deposits_list_base_url}?{urllib.parse.urlencode(params)}" def get_deposits_list(username: Optional[str] = None) -> List[Dict[str, Any]]: """Return the list of software deposits using swh-deposit API """ config = get_config()["deposit"] deposits_list_base_url = config["private_api_url"] + "deposits" deposits_list_auth = HTTPBasicAuth( config["private_api_user"], config["private_api_password"] ) deposits_list_url = _deposits_list_url( deposits_list_base_url, page_size=1, username=username ) nb_deposits = requests.get( deposits_list_url, auth=deposits_list_auth, timeout=30 ).json()["count"] deposits_data = cache.get(f"swh-deposit-list-{username}") if not deposits_data or deposits_data["count"] != nb_deposits: deposits_list_url = _deposits_list_url( deposits_list_base_url, page_size=nb_deposits, username=username ) deposits_data = requests.get( deposits_list_url, auth=deposits_list_auth, timeout=30, ).json() cache.set(f"swh-deposit-list-{username}", deposits_data) return deposits_data["results"] def origin_visit_types() -> List[str]: """Return the exhaustive list of visit types for origins ingested into the archive. """ try: return sorted(search().visit_types_count().keys()) except Exception: return [] def redirect_to_new_route(request, new_route, permanent=True): """Redirect a request to another route with url args and query parameters eg: /origin//log?path=test can be redirected as /log?url=&path=test. This can be used to deprecate routes """ request_path = resolve(request.path_info) args = {**request_path.kwargs, **request.GET.dict()} return redirect(reverse(new_route, query_params=args), permanent=permanent,) diff --git a/swh/web/config.py b/swh/web/config.py index 8b62d009..432b034f 100644 --- a/swh/web/config.py +++ b/swh/web/config.py @@ -1,208 +1,209 @@ # Copyright (C) 2017-2021 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU Affero General Public License version 3, or any later version # See top-level LICENSE file for more information import os from typing import Any, Dict from swh.core import config from swh.counters import get_counters from swh.indexer.storage import get_indexer_storage from swh.scheduler import get_scheduler from swh.search import get_search from swh.storage import get_storage from swh.vault import get_vault from swh.web import settings +SWH_WEB_SERVER_NAME = "archive.softwareheritage.org" SWH_WEB_INTERNAL_SERVER_NAME = "archive.internal.softwareheritage.org" -STAGING_SERVER_NAMES = [ +SWH_WEB_STAGING_SERVER_NAMES = [ "webapp.staging.swh.network", "webapp.internal.staging.swh.network", ] SETTINGS_DIR = os.path.dirname(settings.__file__) DEFAULT_CONFIG = { "allowed_hosts": ("list", []), "storage": ( "dict", {"cls": "remote", "url": "http://127.0.0.1:5002/", "timeout": 10,}, ), "indexer_storage": ( "dict", {"cls": "remote", "url": "http://127.0.0.1:5007/", "timeout": 1,}, ), "counters": ( "dict", {"cls": "remote", "url": "http://127.0.0.1:5011/", "timeout": 1,}, ), "search": ( "dict", {"cls": "remote", "url": "http://127.0.0.1:5010/", "timeout": 10,}, ), "search_config": ( "dict", {"metadata_backend": "swh-indexer-storage",}, # or "swh-search" ), "log_dir": ("string", "/tmp/swh/log"), "debug": ("bool", False), "serve_assets": ("bool", False), "host": ("string", "127.0.0.1"), "port": ("int", 5004), "secret_key": ("string", "development key"), # do not display code highlighting for content > 1MB "content_display_max_size": ("int", 5 * 1024 * 1024), "snapshot_content_max_size": ("int", 1000), "throttling": ( "dict", { "cache_uri": None, # production: memcached as cache (127.0.0.1:11211) # development: in-memory cache so None "scopes": { "swh_api": { "limiter_rate": {"default": "120/h"}, "exempted_networks": ["127.0.0.0/8"], }, "swh_api_origin_search": { "limiter_rate": {"default": "10/m"}, "exempted_networks": ["127.0.0.0/8"], }, "swh_vault_cooking": { "limiter_rate": {"default": "120/h", "GET": "60/m"}, "exempted_networks": ["127.0.0.0/8"], }, "swh_save_origin": { "limiter_rate": {"default": "120/h", "POST": "10/h"}, "exempted_networks": ["127.0.0.0/8"], }, "swh_api_origin_visit_latest": { "limiter_rate": {"default": "700/m"}, "exempted_networks": ["127.0.0.0/8"], }, }, }, ), "vault": ("dict", {"cls": "remote", "args": {"url": "http://127.0.0.1:5005/",}}), "scheduler": ("dict", {"cls": "remote", "url": "http://127.0.0.1:5008/"}), "development_db": ("string", os.path.join(SETTINGS_DIR, "db.sqlite3")), "test_db": ("dict", {"name": "swh-web-test"}), "production_db": ("dict", {"name": "swh-web"}), "deposit": ( "dict", { "private_api_url": "https://deposit.softwareheritage.org/1/private/", "private_api_user": "swhworker", "private_api_password": "some-password", }, ), "e2e_tests_mode": ("bool", False), "es_workers_index_url": ("string", ""), "history_counters_url": ( "string", ( "http://counters1.internal.softwareheritage.org:5011" "/counters_history/history.json" ), ), "client_config": ("dict", {}), "keycloak": ("dict", {"server_url": "", "realm_name": ""}), "graph": ( "dict", { "server_url": "http://graph.internal.softwareheritage.org:5009/graph/", "max_edges": {"staff": 0, "user": 100000, "anonymous": 1000}, }, ), "status": ( "dict", { "server_url": "https://status.softwareheritage.org/", "json_path": "1.0/status/578e5eddcdc0cc7951000520", }, ), "counters_backend": ("string", "swh-storage"), # or "swh-counters" - "staging_server_names": ("list", STAGING_SERVER_NAMES), + "staging_server_names": ("list", SWH_WEB_STAGING_SERVER_NAMES), "instance_name": ("str", "archive-test.softwareheritage.org"), "give": ("dict", {"public_key": "", "token": ""}), } swhweb_config: Dict[str, Any] = {} def get_config(config_file="web/web"): """Read the configuration file `config_file`. If an environment variable SWH_CONFIG_FILENAME is defined, this takes precedence over the config_file parameter. In any case, update the app with parameters (secret_key, conf) and return the parsed configuration as a dict. If no configuration file is provided, return a default configuration. """ if not swhweb_config: config_filename = os.environ.get("SWH_CONFIG_FILENAME") if config_filename: config_file = config_filename cfg = config.load_named_config(config_file, DEFAULT_CONFIG) swhweb_config.update(cfg) config.prepare_folders(swhweb_config, "log_dir") if swhweb_config.get("search"): swhweb_config["search"] = get_search(**swhweb_config["search"]) else: swhweb_config["search"] = None swhweb_config["storage"] = get_storage(**swhweb_config["storage"]) swhweb_config["vault"] = get_vault(**swhweb_config["vault"]) swhweb_config["indexer_storage"] = get_indexer_storage( **swhweb_config["indexer_storage"] ) swhweb_config["scheduler"] = get_scheduler(**swhweb_config["scheduler"]) swhweb_config["counters"] = get_counters(**swhweb_config["counters"]) return swhweb_config def search(): """Return the current application's search. """ return get_config()["search"] def storage(): """Return the current application's storage. """ return get_config()["storage"] def vault(): """Return the current application's vault. """ return get_config()["vault"] def indexer_storage(): """Return the current application's indexer storage. """ return get_config()["indexer_storage"] def scheduler(): """Return the current application's scheduler. """ return get_config()["scheduler"] def counters(): """Return the current application's counters. """ return get_config()["counters"] diff --git a/swh/web/misc/coverage.py b/swh/web/misc/coverage.py index d1649a4f..603a266e 100644 --- a/swh/web/misc/coverage.py +++ b/swh/web/misc/coverage.py @@ -1,393 +1,422 @@ -# Copyright (C) 2018-2021 The Software Heritage developers +# Copyright (C) 2018-2022 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU Affero General Public License version 3, or any later version # See top-level LICENSE file for more information from collections import Counter, defaultdict -from functools import lru_cache -from typing import Dict, List, Tuple +from typing import Any, Dict, List, Tuple from urllib.parse import urlparse import sentry_sdk from django.conf.urls import url +from django.core.cache import cache +from django.http.request import HttpRequest +from django.http.response import HttpResponse from django.shortcuts import render from django.views.decorators.cache import never_cache from django.views.decorators.clickjacking import xframe_options_exempt from swh.scheduler.model import SchedulerMetrics from swh.web.common import archive -from swh.web.common.utils import get_deposits_list, reverse +from swh.web.common.origin_save import get_savable_visit_types +from swh.web.common.utils import get_deposits_list, is_swh_web_production, reverse from swh.web.config import scheduler _swh_arch_overview_doc = ( "https://docs.softwareheritage.org/devel/architecture/overview.html" ) # Current coverage list of the archive in a high level overview fashion, # categorized as follow: # - listed origins: origins discovered using a swh lister # - legacy: origins where public hosting service has closed # - deposited: origins coming from swh-deposit # # TODO: Store that list in a database table somewhere (swh-scheduler, swh-storage ?) # and retrieve it dynamically -listed_origins = { +listed_origins: Dict[str, Any] = { "info": ( "These software origins get continuously discovered and archived using " f'the listers implemented by Software Heritage.' ), "origins": [ { "type": "bitbucket", "info_url": "https://bitbucket.org", "info": "public repositories from Bitbucket", "search_pattern": "https://bitbucket.org/", }, { "type": "cgit", "info_url": "https://git.zx2c4.com/cgit/about", "info": "public repositories from cgit instances", "search_pattern": "cgit", }, { "type": "CRAN", "info_url": "https://cran.r-project.org", "info": "source packages from The Comprehensive R Archive Network", "search_pattern": "https://cran.r-project.org/", }, { "type": "debian", "info_url": "https://www.debian.org", "info": "source packages from Debian and Debian-based distributions", "search_pattern": "deb://", }, { "type": "gitea", "info_url": "https://gitea.io", "info": "public repositories from Gitea instances", "search_pattern": "gitea", }, { "type": "github", "info_url": "https://github.com", "info": "public repositories from GitHub", "search_pattern": "https://github.com/", }, { "type": "gitlab", "info_url": "https://gitlab.com", "info": "public repositories from multiple GitLab instances", "search_pattern": "gitlab", }, { "type": "guix", "info_url": "https://guix.gnu.org", "info": "source code tarballs used to build the Guix package collection", "visit_types": ["nixguix"], "search_pattern": "https://guix.gnu.org/sources.json", }, { "type": "GNU", "info_url": "https://www.gnu.org", "info": "releases from the GNU project (as of August 2015)", "search_pattern": "gnu", }, { "type": "heptapod", "info_url": "https://heptapod.net/", "info": "public repositories from multiple Heptapod instances", "search_pattern": "heptapod", }, { "type": "launchpad", "info_url": "https://launchpad.net", "logo": "img/logos/launchpad.png", "info": "public repositories from Launchpad", "search_pattern": "https://git.launchpad.net/", }, { "type": "nixos", "info_url": "https://nixos.org", "info": "source code tarballs used to build the Nix package collection", "visit_types": ["nixguix"], "search_pattern": ( "https://nix-community.github.io/nixpkgs-swh/sources-unstable.json" ), }, { "type": "npm", "info_url": "https://www.npmjs.com", "info": "public packages from the package registry for javascript", "search_pattern": "https://www.npmjs.com", }, { "type": "opam", "info_url": "https://opam.ocaml.org/", "info": "public packages from the source-based package manager for OCaml", "search_pattern": "opam+https://opam.ocaml.org/", }, # apart our forge, most phabricator origins have not been archived # while they have been listed so do not display those type of origins # until new listing processes have been executed and origins loaded # # { # "type": "phabricator", # "info_url": "https://www.phacility.com/phabricator", # "info": "public repositories from multiple Phabricator instances", # "search_pattern": "phabricator", # }, { "type": "pypi", "info_url": "https://pypi.org", "info": "source packages from the Python Package Index", "search_pattern": "https://pypi.org", }, { "type": "sourceforge", "info_url": "https://sourceforge.net", "info": "public repositories from SourceForge", "search_pattern": "code.sf.net", }, ], } -legacy_origins = { +legacy_origins: Dict[str, Any] = { "info": ( "Discontinued hosting services. Those origins have been archived " "by Software Heritage." ), "origins": [ { "type": "gitorious", "info_url": "https://en.wikipedia.org/wiki/Gitorious", "info": ( "public repositories from the former Gitorious code hosting service" ), "visit_types": ["git"], "search_pattern": "https://gitorious.org", "count": "122,014", }, { "type": "googlecode", "info_url": "https://code.google.com/archive", "info": ( "public repositories from the former Google Code project " "hosting service" ), "visit_types": ["git", "hg", "svn"], "search_pattern": "googlecode.com", "count": "790,026", }, { "type": "bitbucket", "info_url": "https://bitbucket.org", "info": "public repositories from Bitbucket", "search_pattern": "https://bitbucket.org/", "visit_types": ["hg"], "count": "336,795", }, ], } -deposited_origins = { +deposited_origins: Dict[str, Any] = { "info": ( "These origins are directly pushed into the archive by trusted partners " f'using the deposit service of Software Heritage.' ), "origins": [ { "type": "elife", "info_url": "https://elifesciences.org", "info": ( "research software source code associated to the articles " "eLife publishes" ), "search_pattern": "elife.stencila.io", "visit_types": ["deposit"], }, { "type": "hal", "info_url": "https://hal.archives-ouvertes.fr", "info": "scientific software source code deposited in the open archive HAL", "visit_types": ["deposit"], "search_pattern": "hal.archives-ouvertes.fr", }, { "type": "ipol", "info_url": "https://www.ipol.im", "info": "software artifacts associated to the articles IPOL publishes", "visit_types": ["deposit"], "search_pattern": "doi.org/10.5201", }, ], } +_cache_timeout = 5 * 60 -@lru_cache() -def _get_listers_metrics() -> Dict[str, List[Tuple[str, SchedulerMetrics]]]: + +def _get_listers_metrics( + cache_metrics: bool = False, +) -> Dict[str, List[Tuple[str, SchedulerMetrics]]]: """Returns scheduler metrics in the following mapping: Dict[lister_name, List[Tuple[instance_name, SchedulerMetrics]]] as a lister instance has one SchedulerMetrics object per visit type. """ - listers_metrics = defaultdict(list) - try: - listers = scheduler().get_listers() - scheduler_metrics = scheduler().get_metrics() - for lister in listers: - for metrics in filter( - lambda m: m.lister_id == lister.id, scheduler_metrics - ): - listers_metrics[lister.name].append((lister.instance_name, metrics)) - except Exception as e: - sentry_sdk.capture_exception(e) + cache_key = "lister_metrics" + listers_metrics = cache.get(cache_key, {}) + if not listers_metrics: + listers_metrics = defaultdict(list) + try: + listers = scheduler().get_listers() + scheduler_metrics = scheduler().get_metrics() + for lister in listers: + for metrics in filter( + lambda m: m.lister_id == lister.id, scheduler_metrics + ): + listers_metrics[lister.name].append((lister.instance_name, metrics)) + if cache_metrics: + cache.set(cache_key, listers_metrics, timeout=_cache_timeout) + except Exception as e: + sentry_sdk.capture_exception(e) + return listers_metrics -@lru_cache() -def _get_deposits_netloc_counts() -> Counter: +def _get_deposits_netloc_counts(cache_counts: bool = False) -> Counter: """Return deposit counts per origin url network location. """ def _process_origin_url(origin_url): parsed_url = urlparse(origin_url) netloc = parsed_url.netloc # special treatment for doi.org netloc as it is not specific enough # for origins mapping if parsed_url.netloc == "doi.org": netloc += "/" + parsed_url.path.split("/")[1] return netloc - netlocs = [] - try: - deposits = get_deposits_list() - netlocs = [ - _process_origin_url(d["origin_url"]) - for d in deposits - if d["status"] == "done" - ] - except Exception as e: - sentry_sdk.capture_exception(e) - return Counter(netlocs) - - -@lru_cache() -def _get_nixguix_origins_count(origin_url: str) -> int: + cache_key = "deposits_netloc_counts" + deposits_netloc_counts = cache.get(cache_key, Counter()) + if not deposits_netloc_counts: + netlocs = [] + try: + deposits = get_deposits_list() + netlocs = [ + _process_origin_url(d["origin_url"]) + for d in deposits + if d["status"] == "done" + ] + deposits_netloc_counts = Counter(netlocs) + if cache_counts: + cache.set(cache_key, deposits_netloc_counts, timeout=_cache_timeout) + except Exception as e: + sentry_sdk.capture_exception(e) + + return deposits_netloc_counts + + +def _get_nixguix_origins_count(origin_url: str, cache_count: bool = False) -> int: """Returns number of archived tarballs for NixOS, aka the number of branches in a dedicated origin in the archive. """ - snapshot = archive.lookup_latest_origin_snapshot(origin_url) - if snapshot: - snapshot_sizes = archive.lookup_snapshot_sizes(snapshot["id"]) - return snapshot_sizes["release"] - else: - return 0 + cache_key = f"nixguix_origins_count_{origin_url}" + nixguix_origins_count = cache.get(cache_key, 0) + if not nixguix_origins_count: + snapshot = archive.lookup_latest_origin_snapshot(origin_url) + if snapshot: + snapshot_sizes = archive.lookup_snapshot_sizes(snapshot["id"]) + nixguix_origins_count = snapshot_sizes["release"] + else: + nixguix_origins_count = 0 + if cache_count: + cache.set(cache_key, nixguix_origins_count, timeout=_cache_timeout) + return nixguix_origins_count def _search_url(query: str, visit_type: str) -> str: return reverse( "browse-search", query_params={ "q": query, "visit_type": visit_type, "with_visit": "true", "with_content": "true", }, ) @xframe_options_exempt @never_cache -def _swh_coverage(request): - listers_metrics = _get_listers_metrics() +def _swh_coverage(request: HttpRequest) -> HttpResponse: + use_cache = is_swh_web_production(request) + listers_metrics = _get_listers_metrics(use_cache) for origins in listed_origins["origins"]: origins["instances"] = {} origins_type = origins["type"] # special processing for nixos/guix origins as there is no # scheduler metrics for those if origins_type in ("nixos", "guix"): - count = _get_nixguix_origins_count(origins["search_pattern"]) + count = _get_nixguix_origins_count(origins["search_pattern"], use_cache) + origins["count"] = f"{count:,}" if count else "" origins["instances"][origins_type] = {"nixguix": {"count": count}} if origins_type not in listers_metrics: continue count_total = sum( [metrics.origins_known for _, metrics in listers_metrics[origins_type]] ) count_never_visited = sum( [ metrics.origins_never_visited for _, metrics in listers_metrics[origins_type] ] ) count = count_total - count_never_visited origins["count"] = f"{count:,}" origins["instances"] = defaultdict(dict) for instance, metrics in listers_metrics[origins_type]: - # not yet in production - if metrics.visit_type in ("bzr", "cvs"): + # these types are available in staging/docker but not yet in production + if ( + metrics.visit_type in ("bzr", "cvs") + and metrics.visit_type not in get_savable_visit_types() + ): continue instance_count = metrics.origins_known - metrics.origins_never_visited origins["instances"][instance].update( {metrics.visit_type: {"count": f"{instance_count:,}"}} ) origins["visit_types"] = list( set(origins["instances"][instance].keys()) | set(origins.get("visit_types", [])) ) if origins_type == "CRAN": origins["instances"]["cran"]["cran"] = {"count": origins["count"]} # defaultdict cannot be iterated in django template origins["instances"] = dict(origins["instances"]) for origins in listed_origins["origins"]: instances = origins["instances"] nb_instances = len(instances) for instance_name, visit_types in instances.items(): for visit_type in visit_types: if nb_instances > 1: search_pattern = instance_name else: search_pattern = origins["search_pattern"] search_url = _search_url(search_pattern, visit_type) visit_types[visit_type]["search_url"] = search_url for origins in legacy_origins["origins"]: origins["search_urls"] = {} for visit_type in origins["visit_types"]: origins["search_urls"][visit_type] = _search_url( origins["search_pattern"], visit_type ) - deposits_counts = _get_deposits_netloc_counts() + deposits_counts = _get_deposits_netloc_counts(use_cache) + for origins in deposited_origins["origins"]: if origins["search_pattern"] in deposits_counts: origins["count"] = f"{deposits_counts[origins['search_pattern']]:,}" origins["search_urls"] = { "deposit": _search_url(origins["search_pattern"], "deposit") } return render( request, "misc/coverage.html", { "origins": { "Regular crawling": listed_origins, "Discontinued hosting": legacy_origins, "On demand archival": deposited_origins, } }, ) urlpatterns = [ url(r"^coverage/$", _swh_coverage, name="swh-coverage"), ] diff --git a/swh/web/tests/common/test_templatetags.py b/swh/web/tests/common/test_templatetags.py index 1e1e99e3..a3f60025 100644 --- a/swh/web/tests/common/test_templatetags.py +++ b/swh/web/tests/common/test_templatetags.py @@ -1,66 +1,80 @@ # Copyright (C) 2015-2019 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU Affero General Public License version 3, or any later version # See top-level LICENSE file for more information +import pytest + +from swh.web.api.apiresponse import compute_link_header from swh.web.common.swh_templatetags import ( docstring_display, urlize_header_links, urlize_links_and_mails, ) def test_urlize_http_link(): link = "https://example.com/api/1/abc/" expected_content = f'{link}' assert urlize_links_and_mails(link) == expected_content def test_urlize_email(): email = "someone@example.com" expected_content = f'{email}' assert urlize_links_and_mails(email) == expected_content -def test_urlize_header_links(): +@pytest.mark.parametrize( + "next_link, prev_link", + [ + ("https://example.org/api/1/abc/", "https://example.org/api/1/def/"), + ("https://example.org/api/1/0,5/", "https://example.org/api/1/5,10/"), + ], +) +def test_urlize_header_links(next_link, prev_link): - next_link = "https://example.com/api/1/abc/" - prev_link = "https://example.com/api/1/def/" + link_header = f'<{next_link}>; rel="next",<{prev_link}>; rel="previous"' - content = f'<{next_link}>; rel="next"\n<{prev_link}>; rel="prev"' + assert ( + link_header + == compute_link_header( + {"headers": {"link-next": next_link, "link-prev": prev_link}}, options={} + )["Link"] + ) expected_content = ( f'<{next_link}>; rel="next"\n' - f'<{prev_link}>; rel="prev"' + f'<{prev_link}>; rel="previous"' ) - assert urlize_header_links(content) == expected_content + assert urlize_header_links(link_header) == expected_content def test_docstring_display(): # update api link with html links content with links docstring = ( "This is my list header:\n\n" " - Here is item 1, with a continuation\n" " line right here\n" " - Here is item 2\n\n" " Here is something that is not part of the list" ) expected_docstring = ( '
' "

This is my list header:

\n" "
\n" '
    \n' "

  • Here is item 1, with a continuation\n" "line right here

    • \n" "

  • Here is item 2

    • \n" "
\n" "

Here is something that is not part of the list

\n" "
\n" "
" ) assert docstring_display(docstring) == expected_docstring diff --git a/swh/web/tests/common/test_utils.py b/swh/web/tests/common/test_utils.py index dee4d6ac..31593a48 100644 --- a/swh/web/tests/common/test_utils.py +++ b/swh/web/tests/common/test_utils.py @@ -1,299 +1,316 @@ # Copyright (C) 2017-2021 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU Affero General Public License version 3, or any later version # See top-level LICENSE file for more information from base64 import b64encode import datetime from urllib.parse import quote import pytest from django.conf.urls import url from django.test.utils import override_settings from django.urls.exceptions import NoReverseMatch from swh.web.common import utils from swh.web.common.exc import BadInputExc -from swh.web.config import get_config +from swh.web.config import SWH_WEB_SERVER_NAME, SWH_WEB_STAGING_SERVER_NAMES, get_config def test_shorten_path_noop(): noops = ["/api/", "/browse/", "/content/symbol/foobar/"] for noop in noops: assert utils.shorten_path(noop) == noop def test_shorten_path_sha1(): sha1 = "aafb16d69fd30ff58afdd69036a26047f3aebdc6" short_sha1 = sha1[:8] + "..." templates = [ "/api/1/content/sha1:%s/", "/api/1/content/sha1_git:%s/", "/api/1/directory/%s/", "/api/1/content/sha1:%s/ctags/", ] for template in templates: assert utils.shorten_path(template % sha1) == template % short_sha1 def test_shorten_path_sha256(): sha256 = "aafb16d69fd30ff58afdd69036a26047" "213add102934013a014dfca031c41aef" short_sha256 = sha256[:8] + "..." templates = [ "/api/1/content/sha256:%s/", "/api/1/directory/%s/", "/api/1/content/sha256:%s/filetype/", ] for template in templates: assert utils.shorten_path(template % sha256) == template % short_sha256 @pytest.mark.parametrize( "input_timestamp, output_date", [ ( "2016-01-12", datetime.datetime(2016, 1, 12, 0, 0, tzinfo=datetime.timezone.utc), ), ( "2016-01-12T09:19:12+0100", datetime.datetime(2016, 1, 12, 8, 19, 12, tzinfo=datetime.timezone.utc), ), ( "2007-01-14T20:34:22Z", datetime.datetime(2007, 1, 14, 20, 34, 22, tzinfo=datetime.timezone.utc), ), ], ) def test_parse_iso8601_date_to_utc_ok(input_timestamp, output_date): assert utils.parse_iso8601_date_to_utc(input_timestamp) == output_date @pytest.mark.parametrize( "invalid_iso8601_timestamp", ["Today is January 1, 2047 at 8:21:00AM", "1452591542"] ) def test_parse_iso8601_date_to_utc_ko(invalid_iso8601_timestamp): with pytest.raises(BadInputExc): utils.parse_iso8601_date_to_utc(invalid_iso8601_timestamp) def test_format_utc_iso_date(): assert ( utils.format_utc_iso_date("2017-05-04T13:27:13+02:00") == "04 May 2017, 11:27 UTC" ) def test_gen_path_info(): input_path = "/home/user/swh-environment/swh-web/" expected_result = [ {"name": "home", "path": "home"}, {"name": "user", "path": "home/user"}, {"name": "swh-environment", "path": "home/user/swh-environment"}, {"name": "swh-web", "path": "home/user/swh-environment/swh-web"}, ] path_info = utils.gen_path_info(input_path) assert path_info == expected_result input_path = "home/user/swh-environment/swh-web" path_info = utils.gen_path_info(input_path) assert path_info == expected_result def test_rst_to_html(): rst = ( "Section\n" "=======\n\n" "**Some strong text**\n\n" "* This is a bulleted list.\n" "* It has two items, the second\n" " item uses two lines.\n" "\n" "1. This is a numbered list.\n" "2. It has two items too.\n" "\n" "#. This is a numbered list.\n" "#. It has two items too.\n" ) expected_html = ( '

Section

\n' "

Some strong text

\n" '
    \n' "

  • This is a bulleted list.

    • \n" "

  • It has two items, the second\n" "item uses two lines.

    • \n" "
\n" '
    \n' "

  1. This is a numbered list.

    2. \n" "

  2. It has two items too.

    3. \n" "

  3. This is a numbered list.

    4. \n" "

  4. It has two items too.

    5. \n" "
\n" "
" ) assert utils.rst_to_html(rst) == expected_html def sample_test_view(request, string, number): pass def sample_test_view_no_url_args(request): pass urlpatterns = [ url( r"^sample/test/(?P.+)/view/(?P[0-9]+)/$", sample_test_view, name="sample-test-view", ), url( r"^sample/test/view/no/url/args/$", sample_test_view_no_url_args, name="sample-test-view-no-url-args", ), ] @override_settings(ROOT_URLCONF=__name__) def test_reverse_url_args_only_ok(): string = "foo" number = 55 url = utils.reverse( "sample-test-view", url_args={"string": string, "number": number} ) assert url == f"/sample/test/{string}/view/{number}/" @override_settings(ROOT_URLCONF=__name__) def test_reverse_url_args_only_ko(): string = "foo" with pytest.raises(NoReverseMatch): utils.reverse("sample-test-view", url_args={"string": string, "number": string}) @override_settings(ROOT_URLCONF=__name__) def test_reverse_no_url_args(): url = utils.reverse("sample-test-view-no-url-args") assert url == "/sample/test/view/no/url/args/" @override_settings(ROOT_URLCONF=__name__) def test_reverse_query_params_only(): start = 0 scope = "foo" url = utils.reverse( "sample-test-view-no-url-args", query_params={"start": start, "scope": scope} ) assert url == f"/sample/test/view/no/url/args/?scope={scope}&start={start}" url = utils.reverse( "sample-test-view-no-url-args", query_params={"start": start, "scope": None} ) assert url == f"/sample/test/view/no/url/args/?start={start}" @override_settings(ROOT_URLCONF=__name__) def test_reverse_query_params_encode(): libname = "libstc++" url = utils.reverse( "sample-test-view-no-url-args", query_params={"libname": libname} ) assert url == f"/sample/test/view/no/url/args/?libname={quote(libname, safe='/;:')}" @override_settings(ROOT_URLCONF=__name__) def test_reverse_url_args_query_params(): string = "foo" number = 55 start = 10 scope = "bar" url = utils.reverse( "sample-test-view", url_args={"string": string, "number": number}, query_params={"start": start, "scope": scope}, ) assert url == f"/sample/test/{string}/view/{number}/?scope={scope}&start={start}" @override_settings(ROOT_URLCONF=__name__) def test_reverse_absolute_uri(request_factory): request = request_factory.get(utils.reverse("sample-test-view-no-url-args")) url = utils.reverse("sample-test-view-no-url-args", request=request) assert url == f"http://{request.META['SERVER_NAME']}/sample/test/view/no/url/args/" def test_get_deposits_list(requests_mock): deposits_data = { "count": 2, "results": [ { "check_task_id": "351820217", "client": 2, "collection": 1, "complete_date": "2021-01-21T07:52:19.919312Z", "external_id": "hal-03116143", "id": 1412, "load_task_id": "351820260", "origin_url": "https://hal.archives-ouvertes.fr/hal-03116143", "parent": None, "reception_date": "2021-01-21T07:52:19.471019Z", "status": "done", "status_detail": None, "swhid": "swh:1:dir:f25157ad1b13cb20ac3457d4f6756b49ac63d079", }, { "check_task_id": "381576507", "client": 2, "collection": 1, "complete_date": "2021-07-07T08:00:44.726676Z", "external_id": "hal-03275052", "id": 1693, "load_task_id": "381576508", "origin_url": "https://hal.archives-ouvertes.fr/hal-03275052", "parent": None, "reception_date": "2021-07-07T08:00:44.327661Z", "status": "done", "status_detail": None, "swhid": "swh:1:dir:825fa96d1810177ec08a772ffa5bd34bbd08b89c", }, ], } config = get_config()["deposit"] deposits_list_url = config["private_api_url"] + "deposits" basic_auth_payload = ( config["private_api_user"] + ":" + config["private_api_password"] ).encode() requests_mock.get( deposits_list_url, json=deposits_data, request_headers={ "Authorization": f"Basic {b64encode(basic_auth_payload).decode('ascii')}" }, ) assert utils.get_deposits_list() == deposits_data["results"] @pytest.mark.parametrize("backend", ["swh-search", "swh-storage"]) def test_origin_visit_types(mocker, backend): if backend != "swh-search": # equivalent to not configuring search in the config search = mocker.patch("swh.web.common.utils.search") search.return_value = None assert utils.origin_visit_types() == [] else: # see swh/web/tests/data.py for origins added for tests assert utils.origin_visit_types() == ["git", "tar"] + + +@pytest.mark.parametrize("server_name", ["localhost", "127.0.0.1", "testserver"]) +def test_is_swh_web_development(request_factory, server_name): + request = request_factory.get("/", SERVER_NAME=server_name) + assert utils.is_swh_web_development(request) + + +@pytest.mark.parametrize("server_name", SWH_WEB_STAGING_SERVER_NAMES) +def test_is_swh_web_staging(request_factory, server_name): + request = request_factory.get("/", SERVER_NAME=server_name) + assert utils.is_swh_web_staging(request) + + +def test_is_swh_web_production(request_factory): + request = request_factory.get("/", SERVER_NAME=SWH_WEB_SERVER_NAME) + assert utils.is_swh_web_production(request) diff --git a/swh/web/tests/misc/test_coverage.py b/swh/web/tests/misc/test_coverage.py index 1de2b2c7..f35501ff 100644 --- a/swh/web/tests/misc/test_coverage.py +++ b/swh/web/tests/misc/test_coverage.py @@ -1,133 +1,131 @@ -# Copyright (C) 2021 The Software Heritage developers +# Copyright (C) 2021-2022 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU Affero General Public License version 3, or any later version # See top-level LICENSE file for more information from datetime import datetime, timezone from itertools import chain import os -from random import choice, randint +from random import randint import uuid -import pytest - from django.conf import settings from django.utils.html import escape from swh.scheduler.model import LastVisitStatus, ListedOrigin, OriginVisitStats from swh.web.common.utils import reverse -from swh.web.misc.coverage import ( - _get_deposits_netloc_counts, - _get_listers_metrics, - deposited_origins, - legacy_origins, - listed_origins, -) +from swh.web.config import SWH_WEB_SERVER_NAME +from swh.web.misc.coverage import deposited_origins, legacy_origins, listed_origins from swh.web.tests.django_asserts import assert_contains -from swh.web.tests.utils import check_html_get_response - +from swh.web.tests.utils import check_html_get_response, check_http_get_response -@pytest.fixture(autouse=True) -def clear_lru_caches(): - _get_listers_metrics.cache_clear() - _get_deposits_netloc_counts.cache_clear() - -def test_coverage_view_no_metrics(client): +def test_coverage_view_no_metrics(client, swh_scheduler): """ Check coverage view can be rendered when scheduler metrics and deposits data are not available. """ url = reverse("swh-coverage") check_html_get_response( client, url, status_code=200, template_used="misc/coverage.html" ) def test_coverage_view_with_metrics(client, swh_scheduler, mocker): """ Generate some sample scheduler metrics and some sample deposits that will be consumed by the archive coverage view, then check the HTML page gets rendered without errors. """ - mocker.patch( - "swh.web.misc.coverage._get_nixguix_origins_count" - ).return_value = 30095 + + # mock calls to get nixguix origin counts + mock_archive = mocker.patch("swh.web.misc.coverage.archive") + mock_archive.lookup_latest_origin_snapshot.return_value = {"id": "some-snapshot"} + mock_archive.lookup_snapshot_sizes.return_value = {"release": 30095} + listers = [] + visit_types = ["git", "hg", "svn", "bzr", "svn"] for origins in listed_origins["origins"]: # create some instances for each lister for instance in range(randint(1, 5)): lister = swh_scheduler.get_or_create_lister( origins["type"], f"instance-{instance}" ) listers.append(lister) # record some sample listed origins _origins = [] origin_visit_stats = [] - for i in range(randint(3, 10)): + for i, visit_type in enumerate(visit_types): url = str(uuid.uuid4()) - visit_type = choice(["git", "hg", "svn"]) _origins.append( ListedOrigin( lister_id=lister.id, url=url, visit_type=visit_type, extra_loader_arguments={}, ) ) # set origin visit stats to some origins if i % 2 == 0: now = datetime.now(tz=timezone.utc) origin_visit_stats.append( OriginVisitStats( url=url, visit_type=visit_type, last_successful=now, last_visit=now, last_visit_status=LastVisitStatus.successful, last_snapshot=os.urandom(20), ) ) # send origins data to scheduler swh_scheduler.record_listed_origins(_origins) swh_scheduler.origin_visit_stats_upsert(origin_visit_stats) # compute scheduler metrics swh_scheduler.update_metrics() # add some sample deposits deposits = [] for origins in deposited_origins["origins"]: for _ in range(randint(2, 10)): deposits.append( { "origin_url": f"https://{origins['search_pattern']}/{uuid.uuid4()}", "status": "done", } ) get_deposits_list = mocker.patch("swh.web.misc.coverage.get_deposits_list") get_deposits_list.return_value = deposits # check view gets rendered without errors url = reverse("swh-coverage") resp = check_html_get_response( client, url, status_code=200, template_used="misc/coverage.html" ) # check logos and origins search links are present in the rendered page for origins in chain( listed_origins["origins"], legacy_origins["origins"], deposited_origins["origins"], ): logo_url = f'{settings.STATIC_URL}img/logos/{origins["type"].lower()}.png' assert_contains(resp, f'src="{logo_url}"') if "instances" in origins: - for visit_types in origins["instances"].values(): - for data in visit_types.values(): + for visit_types_ in origins["instances"].values(): + for data in visit_types_.values(): if data["count"]: assert_contains(resp, f'{visit_type}") + + # check request as in production with cache enabled + check_http_get_response( + client, url, status_code=200, server_name=SWH_WEB_SERVER_NAME + ) diff --git a/swh/web/tests/test_templates.py b/swh/web/tests/test_templates.py index a053a4ce..10f53ac2 100644 --- a/swh/web/tests/test_templates.py +++ b/swh/web/tests/test_templates.py @@ -1,91 +1,96 @@ # Copyright (C) 2021 The Software Heritage developers # See the AUTHORS file at the top-level directory of this distribution # License: GNU Affero General Public License version 3, or any later version # See top-level LICENSE file for more information from copy import deepcopy import random from pkg_resources import get_distribution import pytest from swh.web.auth.utils import ADMIN_LIST_DEPOSIT_PERMISSION from swh.web.common.utils import reverse -from swh.web.config import STAGING_SERVER_NAMES, get_config +from swh.web.config import SWH_WEB_SERVER_NAME, SWH_WEB_STAGING_SERVER_NAMES, get_config from swh.web.tests.django_asserts import assert_contains, assert_not_contains from swh.web.tests.utils import check_http_get_response, create_django_permission swh_web_version = get_distribution("swh.web").version def test_layout_without_ribbon(client): url = reverse("swh-web-homepage") - resp = check_http_get_response(client, url, status_code=200) + resp = check_http_get_response( + client, url, status_code=200, server_name=SWH_WEB_SERVER_NAME + ) assert_not_contains(resp, "swh-corner-ribbon") def test_layout_with_staging_ribbon(client): url = reverse("swh-web-homepage") resp = check_http_get_response( - client, url, status_code=200, server_name=random.choice(STAGING_SERVER_NAMES), + client, + url, + status_code=200, + server_name=random.choice(SWH_WEB_STAGING_SERVER_NAMES), ) assert_contains(resp, "swh-corner-ribbon") assert_contains(resp, f"Staging
v{swh_web_version}") def test_layout_with_development_ribbon(client): url = reverse("swh-web-homepage") resp = check_http_get_response( client, url, status_code=200, server_name="localhost", ) assert_contains(resp, "swh-corner-ribbon") assert_contains(resp, f"Development
v{swh_web_version.split('+')[0]}") def test_layout_with_oidc_auth_enabled(client): url = reverse("swh-web-homepage") resp = check_http_get_response(client, url, status_code=200) assert_contains(resp, reverse("oidc-login")) def test_layout_without_oidc_auth_enabled(client, mocker): config = deepcopy(get_config()) config["keycloak"]["server_url"] = "" mock_get_config = mocker.patch("swh.web.common.utils.get_config") mock_get_config.return_value = config url = reverse("swh-web-homepage") resp = check_http_get_response(client, url, status_code=200) assert_contains(resp, reverse("login")) def test_layout_swh_web_version_number_display(client): url = reverse("swh-web-homepage") resp = check_http_get_response(client, url, status_code=200) assert_contains(resp, f"swh-web v{swh_web_version}") @pytest.mark.django_db def test_layout_no_deposit_admin_for_anonymous_user(client): url = reverse("swh-web-homepage") resp = check_http_get_response(client, url, status_code=200) assert_not_contains(resp, "swh-deposit-admin-link") @pytest.mark.django_db def test_layout_deposit_admin_for_staff_user(client, staff_user): client.force_login(staff_user) url = reverse("swh-web-homepage") resp = check_http_get_response(client, url, status_code=200) assert_contains(resp, "swh-deposit-admin-link") @pytest.mark.django_db def test_layout_deposit_admin_for_user_with_permission(client, regular_user): regular_user.user_permissions.add( create_django_permission(ADMIN_LIST_DEPOSIT_PERMISSION) ) client.force_login(regular_user) url = reverse("swh-web-homepage") resp = check_http_get_response(client, url, status_code=200) assert_contains(resp, "swh-deposit-admin-link")